How to fix: ineffective mark-compacts near heap limit allocation failed - JavaScript heap out of memory

JavaScript's heap out of memory error typically occurs when the Node.js environment runs out of memory during garbage collection due to heavy memory usage or memory leaks. This guide provides actionable solutions for engineers to troubleshoot and resolve this error.

Understanding the error

When Node.js applications consume more memory than what is available, V8 triggers garbage collection to free up space. If this process does not recover enough memory, the ineffective mark-compacts near heap limit allocation failed error is thrown, signaling that the JavaScript heap is out of memory.

Checking memory limits

Node.js has a default memory limit. Use the process.memoryUsage() method to monitor memory usage:

console.log(process.memoryUsage());

Increasing memory limits

To temporarily increase the memory limit, use the --max-old-space-size flag when starting your Node.js application:

node --max-old-space-size=4096 your-script.js

Replace 4096 with the number of megabytes you want to allocate.

Profiling memory usage

Identify memory leaks by taking and analyzing heap snapshots with tools like Chrome DevTools or the heapdump module. Profile memory usage by:

const heapdump = require('heapdump');
heapdump.writeSnapshot((err, filename) => {
  console.log('Heap dump written to', filename);
});

Optimizing code

Review your code for common memory leaks, such as unintended global variables, forgotten timers, or closures that hold onto outer scope variables. Refactor the code to ensure variables are not being retained unnecessarily.

Using memory management techniques

Apply memory management techniques such as debouncing and throttling for event listeners, using weak references with WeakMap or WeakSet, and explicitly setting variables to null after use.

Streamlining data processing

For applications handling large datasets, use streaming data processing with stream module to process data in chunks rather than loading it entirely in memory.

const fs = require('fs');
const readStream = fs.createReadStream('large-file.txt');
readStream.on('data', (chunk) => {
  // Process chunk
});

Leveraging external storage

Offload sessions and other memory-intensive data to external stores like Redis or databases. This reduces the memory footprint on the Node.js process.

Modularizing the application

Break down your Node.js application into microservices or child processes using the cluster module to distribute memory load.

Applying garbage collection strategies

Force garbage collection at strategic points in your application if you're running Node.js with --expose-gc flag:

if (global.gc) {
  global.gc();
} else {
  console.log('Garbage collection is not exposed');
}