Looks like it has something to do with internal String handling. I took a heap dump both with 9.1.2.0 shortly after the problematic file and with 9.1.3.0 shortly before the OOM Exception:

9.1.2.0: 194.000 Strings have 194.000 char[] with 15 MB:

9.1.3.0: 196.000 Strings have 196.000 char[] with 220 MB:

I took a screenshot of one example char[] list, but I guess it's not very useful. If I should send you the heap dump I need a location:

The heap dump would be helpful. It's not clear what those strings are from, but I'd need to look at who is holding a reference to them. My first thought would be something changed in our Java integration that's holding too many string keys now, but I don't think we ever cached anything using Java bytecode signatures like (Ljava/util/function/Consumer....

Oh and as far as sending the heap dump...

Easiest is probably to zip it up and post it on Dropbox or Google Drive or something and share a link with us. Time is of the essence...we'd like to investigate this and spin a 9.1.4.0 release ASAP.

You have a link in your inbox.

so far ... I see 2 StringBuilder instances (locals) holding up to 500MB the bigger is 400MB starts with content such as: #<Objectfab::CisGen::Dsl::Model:0x5b22f3ac @name="Freigabe", @presenters={"pr_freigabe_info"=>#<Objectfab::CisGen::Dsl::Presenter:0x671d1b4d @create=#<Objectfab::CisGen::Dsl::Action:0x5ca92647 @names={}, @is_user_generated=true, @parent ...
... so it seems to me like were chasing an application specific leak for you.
are you sure the setup is the very same with 9.1.2 ?

As I wrote, the generator itself is very resource-intense, yes.

But I ran it today several times with 9.0.5.0 and 9.1.3.0 in the same project. The only 2 things I changed were the "ruby"-line in the Gemfile and the "rbenv shell" command

@MartinKoerner Can you get us a dump of the same app under 9.1.2.0? It would allow us to diff the two and see what changed to consume more memory.

@MartinKoerner Ok, a couple things, now that I'm looking at your largest heap dump.

I see what @kares sees...the largest object in the system is a char[] containing what looks like inspect output from one of your model objects. However I don't think this is necessarily an application issue. For example, a NameError raised from a large object graph will in some cases end up inspecting that graph and holding on to a giant string.

What I can't tell is where this StringBuilder came from or what it's being used for. In the largest dump, it appears to be in a JVM local variable, and I can't see what the code using that variable is doing.

A couple suggestions and questions:

• Are you sure the app uses more memory? Check process size for both. I'm wondering if our default heap cap logic is screwed up in some way on 9.1.3.0.
• If you determine that the app really is using more memory, figure out how much more you need to give it to run (using -Xmx####M or -J-Xmx####M to set max heap in MB). If no amount of memory works, we have a leak. If you can get it to work reasonably with a larger heap, we may just have something that got bigger (maybe in error).
• Try running without did_you_mean by passing --disable-did_you_mean to JRuby. This decorates NameError with additional logic, and it's possible that's retaining something it shouldn't.

I will keep poking around the dump I have and try to find more clues.

It looks like the big StringBuilder is in response to a Ruby call to inspect some object graph that has a singleton Objectfab::CisGen::Dsl::Model object. Is it expected in your app that inspecting one of these models would produce a 400MB string?

The top-level object that triggers the inspect appears to be a ObjectFab::CisGen::Dsl::Presenter, FWIW.

Here's a patch that would reduce the memory needed for inspect by going with ByteList right away: https://gist.github.com/headius/59e19a3a4a45c63b06cc9876a8148e28

This isn't a fix but it would help this large inspect fit in memory.

Since we need to put out 9.1.4.0 for a couple other regressions in 9.1.3.0, we're probably going to have to punt this to 9.1.5.0 and work with you to figure it out.

• disabling --disable-did_you_mean did not change anything
• changing Memory usage to "-J-Xms8g -J-Xmx8g" made the waiting time longer but in the end did noch change anything
• the codebase is somewhat old and "inspect" was indeed causing troubles for debugging outputs sometime. But it isn't used for the normal behavior
  • I took all ObjectFab::DSL classes and gave them a simple def inspect; "#{self.class}#{object_id}"; end
  • So I got the real exception:

NoMethodError: protected method `destination=' called for #
Did you mean?  destination
           initialize at /usr/local/var/rbenv/versions/jruby-9.1.3.0/lib/ruby/gems/shared/gems/thor-0.19.1/lib/thor/actions/empty_directory.rb:36
           initialize at /usr/local/var/rbenv/versions/jruby-9.1.3.0/lib/ruby/gems/shared/gems/thor-0.19.1/lib/thor/actions/create_file.rb:37
          create_file at /usr/local/var/rbenv/versions/jruby-9.1.3.0/lib/ruby/gems/shared/gems/thor-0.19.1/lib/thor/actions/create_file.rb:25
             template at /Users/martin/RubymineProjects/cis/lib/generators/objectfab/lib/merged_markers/marker_merge.rb:49

So it looks like a Thor exception, but - I'm using the same Thor 0.19.1 with the other JRuby versions (I double checked the code) - and the method destination= is there! It's a protected level method though...

So is it related with this bug? #2198

I sent you the heap for the 9.1.2.0 run in case you still find it helpful.

Unfortunately, I won't be able to investigate further until next Tuesday...

So, I got a little bit further:

1. The memory issue comes from a circular dependence in the inspecting code. Sorry, that I bothered you with that.
2. I still don't know, why the inspecting is started while printing the "NoMethodError", even if did_you_mean is disabled
3. The Exception is only thrown, if the "--dev" flag is active. If I remove it, the generator runs correctly. All other starting-related flags seem to be no problem

@headius should I open new tickets for 2 and 3? Or simply rename this one (the OOM is not the problem anymore)

@MartinKoerner Definitely file one or two new issues depending on how you feel about the other items. Our process is that bugs like this one get closed as "Invalid" so we don't get confusing context.