Symptoms that are not always leaks
RSS climbing under load can be V8 holding onto heap after a burst, buffers in the kernel, or a real leak. I do not declare a leak until memory keeps rising across idle periods or after forced GC, and request rates are steady.
First metrics: process.memoryUsage(), container memory limit events, and GC pause time if you expose it. If heapUsed sawtooths but returns to a baseline after idle, you may just need a larger heap or better caching policy—not a hunt for retainers.
setInterval(() => {
const m = process.memoryUsage();
console.log(JSON.stringify({
ts: new Date().toISOString(),
rss: m.rss,
heapTotal: m.heapTotal,
heapUsed: m.heapUsed,
external: m.external,
}));
}, 30000);
Reproduce before you snapshot
I build a script that hits the suspected route in a loop with realistic payloads. Without a repro, snapshots show noise. With a repro, the retained size delta between snapshot A and B screams.
Run against a staging instance with production-like data volume. Tiny fixtures hide leaks in caches keyed by customer id.
Keep the repro script in the repo under scripts/leak-repro.js so the next engineer does not reinvent load generation.
Taking heap snapshots
For local debugging I start Node with --inspect and use Chrome DevTools Memory tab. Take a snapshot at idle baseline, run the load script, force a minor wait, take another snapshot, then compare.
In production-like containers I use heapdump or the inspector protocol behind an authenticated admin endpoint that only ops can hit. Never expose unauthenticated snapshot endpoints.
# Local
node --inspect=0.0.0.0:9229 server.js
# Or write a snapshot on demand (protect this route)
import v8 from 'node:v8';
import fs from 'node:fs';
app.post('/admin/heapdump', requireAdmin, (req, res) => {
const file = `/tmp/heap-${Date.now()}.heapsnapshot`;
const snapshotStream = v8.writeHeapSnapshot(file);
res.json({ file: snapshotStream });
});
Reading the comparison
In DevTools, compare snapshots and sort by retained size delta. Look for arrays, maps, and detached DOM-like structures—though on the server it is usually Maps, arrays of request context, or sockets.
Click through retainer chains. Ask: who still points at this object? Common answers: a module-level Map, an EventEmitter listener list, a global cache without TTL, or a closure holding the entire Express req.
Leaks I keep finding
Unbounded caches: const cache = new Map() keyed by userId with no eviction. Fix with LRU and a max size, or Redis with TTL.
EventEmitter listener leaks: subscribe per request to a shared bus and forget to unsubscribe. MaxListeners warnings are a gift—do not raise the limit to silence them without fixing the cause.
Timers and intervals created per connection that never clear. WebSocket servers are frequent offenders.
Third-party SDKs sometimes register global handlers. If retained size points into node_modules, upgrade or isolate the SDK in a worker process.
// Leaky pattern
const sessions = new Map();
app.post('/login', (req, res) => {
sessions.set(req.body.userId, { req, huge: req.body }); // retains request forever
});
// Better: store ids and minimal state, bound the map
import { LRUCache } from 'lru-cache';
const sessions = new LRUCache({ max: 5000, ttl: 1000 * 60 * 30 });
app.post('/login', (req, res) => {
sessions.set(req.body.userId, { userId: req.body.userId, role: 'user' });
});
Native memory and external buffers
Sometimes heapUsed looks stable while RSS climbs. That can be Buffer allocations, native addons, or compression libraries. Check process.memoryUsage().external and tool-specific metrics.
Streaming large uploads into memory is a classic. Prefer streaming to disk or object storage. I once “temporarily” collected upload chunks in an array for “just image uploads” and then a client sent 400MB videos.
Fix, verify, prevent
After a fix, re-run the same load script and confirm the second snapshot no longer grows the hot objects. Add a memory ceiling alert in staging that fails the deploy if heapUsed trends up across a soak test.
Code review flags: module-level collections, addListener without removeListener, setInterval without clear, and caching user content without bounds.
Clinic, 0x, and when DevTools is enough
Chrome DevTools snapshots handle most app-level leaks. For CPU paired with memory weirdness I reach for clinic heapprofiler or doctor. They add overhead—use them in staging soak tests, not on every production pod.
If the leak only appears after hours, schedule snapshot capture at +10m, +60m, +180m under synthetic load. Comparing three points shows growth rate and whether a particular constructor keeps climbing.
Document the finding in the ticket with before/after retained sizes. Future you will thank present you when the same Map pattern returns in another service.
Promises, closures, and the quiet retainers
Long-lived promises that close over request-scoped objects keep those objects alive until the promise settles. A dangling retry loop holding the original req body is a leak shaped like “we are being careful about retries.”
Async queues that buffer failed jobs in memory without a max length will grow forever during a downstream outage. Bound the queue. Shed load. Prefer durable queues for work that must survive process restart anyway.
- Bound in-memory retry buffers
- Avoid closing over full req/res in timers
- Snapshot on a schedule during soak tests
- Record retained-size deltas in the fix PR
A walkthrough of one real leak
We had a notification service whose heap grew ~15MB per hour under steady load. Snapshots showed thousands of Discord webhook payload strings retained by an array on a module-level Metrics object that pushed every outbound body “for debugging.”
The fix was three lines: stop retaining bodies, keep only counters, and sample errors to the logger. Heap stabilized within minutes of deploy. The expensive part was not the fix—it was proving the retainer with two snapshots so nobody argued about “maybe Redis.”
Takeaway I repeat in postmortems: if you add a debug collector, give it a max length on day one. Debug code becomes production code the moment it ships.
Production safety while debugging
Heap snapshots are large and CPU-heavy. Taking them on every pod during peak traffic can cause the outage you are investigating. Prefer a single canary instance, off-peak windows, or staging soak with production traffic replay.
Strip secrets from snapshots before sharing with vendors or pasting into tickets. Snapshots can contain strings from requests, including tokens.
After remediation, leave the metrics and alerts in place. Leaks return in new features that copy old patterns.
- Snapshot canaries, not the whole fleet at once
- Treat heapsnapshot files as sensitive
- Keep memory trend alerts after the fix
- Cap every new in-process collector
Habits that prevent the next leak
Code review questions I ask: Does this Map or array live at module scope? What is its maximum size? Who clears this interval? Does this listener die with the socket?
Load tests should include a memory assertion: after N minutes of steady traffic and a short idle, heapUsed must not exceed baseline by more than a set percentage. CI will not catch every leak, but it catches the loud ones before customers do.
When you vendor a new SDK, read its README for global state and debugging flags. Many “enable verbose logging” options retain payloads. Keep them off in production.
Heap snapshots look intimidating once. After two investigations they become a normal tool—like reading a stack trace. Invest in that skill; it pays back the first time RSS graphs only go up and to the right.
Key takeaways
- Confirm a leak with steady load plus rising heap across idle/GC—not a single RSS spike.
- Reproduce with a script before taking snapshots so comparisons are meaningful.
- Compare two heap snapshots and follow retainer chains to the owning cache or listener.
- Bound in-process caches; prefer TTL/LRU over unbounded Maps.
- Unsubscribe listeners and clear timers per connection lifecycle.
- Guard heapdump endpoints and add soak-test memory alerts to catch regressions.
About the author
Ram — Founder & Editor, BudhiWorks. I build and ship production web apps — Node, React/Next.js, Postgres, and the boring infrastructure that keeps them online. BudhiWorks is where I publish the guides I wish I had when something broke at 2 a.m.