Please see: #5465

So the output your seeing from running 0.11.2 translates as 133943296 = ~128 MB. That's completely expected and simply has to do with how more current v8 handles garbage collection. As you can see from your top two graphs, v8 in v0.8.x would mark-sweep the Persistent objects created for Buffers much more often. Now, if the memory is available, v8 will delay those for better performance.

Though there is another problem your tests brought up. I dropped the logging and ran the test every millisecond for https.js (against master) with --max_old_space_size=32. Here's what happened:

$ /usr/bin/time ./master/node --max_old_space_size=32 server.js
node: ../src/node_crypto.cc:986: void node::crypto::Connection::ClearError(): Assertion `handle_->Get(String::New("error"))->BooleanValue() == false' failed.
Command terminated by signal 6
299.99user 14.30system 6:33.94elapsed 79%CPU (0avgtext+0avgdata 139876maxresident)k
0inputs+0outputs (0major+363398minor)pagefaults 0swaps

/cc @indutny @bnoordhuis

@trevnorris Ok, thanks for taking a look. I'll have to digest this a bit though, it definitely throws a bit of a curveball my way.

In my case I'm working on a thin monitoring agent that periodically reports to an API. Ideally I'm able to do this in the manner that has the least impact on the profile of the code being monitored. I guess where in the 0.8 world I may have been triggering more GC activity, in the 0.10+ world I'm consuming more memory.

In this less frequent mark-sweep world what is a good rule of thumb for observing memory issues over time? Seeing that the minimum heapUsed bounces off a fairly stable point every few GCs?

@brycebaril the memory leaks you need to worry about are when a Buffer sticks around. If the v8 heap fills then node will just crash. Which isn't great, but in the long run not horrible. Since Buffers use external memory, they can really FUBAR your machine by filling up all memory. These can be analyzed by a heap dump. Just search for "heap dump" at npmjs.org.

@trevnorris Strange, I can't reproduce that.

@brycebaril I second what @trevnorris said and will add that part of the problem with your test case is that the C heap tends to get badly fragmented over time. (Plus, most malloc implementations are not very eager to return memory to the operating system even if they could - system calls are expensive.)

When I run your server.js script on an x86_64 machine with --max_old_space_size=32, it maxes out around 170 MB of which about 60 MB is "lost" (at least temporarily) due to fragmentation. I don't think there's much we can do about that because the bulk of the malloc/free calls originate inside openssl.

I'm going to tentatively close this as NOTABUG. If you have reason to believe otherwise, let me know.

@bnoordhuis Here's some more information:

/tmp/node_leak_test/https.js:5
  var m = process.memoryUsage();
                  ^
Error: EMFILE, too many open files
    at sendMemory (/tmp/node_leak_test/https.js:5:19)
    at wrapper [as _onTimeout] (timers.js:252:14)
    at Timer.listOnTimeout [as ontimeout] (timers.js:110:15)

and

node: ../src/node_crypto.cc:993: void node::crypto::Connection::ClearError(): Assertion `handle_->Get(String::New("error"))->BooleanValue() == false' failed.

Don't know exactly why, but when I remove process.memoryUsage() I can't get it to crash. Any ideas off the top of your head?

I think that's a small bug in libuv's darwin.c. It checks errno when task_info() returns something other than KERN_SUCCESS but I'm reasonably sure that function (like other mach functions) doesn't touch errno at all. IOW, you're probably seeing the wrong error.

What happens when you apply joyent/libuv@a1cb52a? If you manage to hit the assert I added, can you inspect the value of err in gdb?

for posterity, this was on linux:

<bnoordhuis> on linux, it tries to open /proc/self/stat - so yes, that can fail with EMFILE