-
-
Notifications
You must be signed in to change notification settings - Fork 925
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
OutOfMemoryError with 9.1.3.0 #4127
Comments
The heap dump would be helpful. It's not clear what those strings are from, but I'd need to look at who is holding a reference to them. My first thought would be something changed in our Java integration that's holding too many string keys now, but I don't think we ever cached anything using Java bytecode signatures like |
Oh and as far as sending the heap dump... Easiest is probably to zip it up and post it on Dropbox or Google Drive or something and share a link with us. Time is of the essence...we'd like to investigate this and spin a 9.1.4.0 release ASAP. |
You have a link in your inbox. |
so far ... I see 2 |
As I wrote, the generator itself is very resource-intense, yes. But I ran it today several times with 9.0.5.0 and 9.1.3.0 in the same project. The only 2 things I changed were the "ruby"-line in the Gemfile and the "rbenv shell" command |
@MartinKoerner Can you get us a dump of the same app under 9.1.2.0? It would allow us to diff the two and see what changed to consume more memory. |
@MartinKoerner Ok, a couple things, now that I'm looking at your largest heap dump. I see what @kares sees...the largest object in the system is a char[] containing what looks like inspect output from one of your model objects. However I don't think this is necessarily an application issue. For example, a NameError raised from a large object graph will in some cases end up inspecting that graph and holding on to a giant string. What I can't tell is where this StringBuilder came from or what it's being used for. In the largest dump, it appears to be in a JVM local variable, and I can't see what the code using that variable is doing. A couple suggestions and questions:
I will keep poking around the dump I have and try to find more clues. |
It looks like the big StringBuilder is in response to a Ruby call to inspect some object graph that has a singleton |
The top-level object that triggers the inspect appears to be a |
Here's a patch that would reduce the memory needed for inspect by going with ByteList right away: https://gist.github.com/headius/59e19a3a4a45c63b06cc9876a8148e28 This isn't a fix but it would help this large inspect fit in memory. |
Since we need to put out 9.1.4.0 for a couple other regressions in 9.1.3.0, we're probably going to have to punt this to 9.1.5.0 and work with you to figure it out. |
This should reduce the overhead of inspecting by avoiding the intermediate StringBuilder/char[] and subsequent conversion to byte[]. There are a few extra objects created along the way (java.lang.String.getBytes, RubyString for class name, etc) but overall memory use should reduce around 3x: a US-ASCII string would need 2x byte size in the StringBuilder, and the builder is still alive when we create the eventual byte[] for a total of 3x max memory use during inspect. Relates to jruby#4127, but does not constitute a fix for memory issues there.
NoMethodError: protected method `destination=' called for # Did you mean? destination initialize at /usr/local/var/rbenv/versions/jruby-9.1.3.0/lib/ruby/gems/shared/gems/thor-0.19.1/lib/thor/actions/empty_directory.rb:36 initialize at /usr/local/var/rbenv/versions/jruby-9.1.3.0/lib/ruby/gems/shared/gems/thor-0.19.1/lib/thor/actions/create_file.rb:37 create_file at /usr/local/var/rbenv/versions/jruby-9.1.3.0/lib/ruby/gems/shared/gems/thor-0.19.1/lib/thor/actions/create_file.rb:25 template at /Users/martin/RubymineProjects/cis/lib/generators/objectfab/lib/merged_markers/marker_merge.rb:49 So it looks like a Thor exception, but - I'm using the same Thor 0.19.1 with the other JRuby versions (I double checked the code) - and the method |
So is it related with this bug? #2198 |
I sent you the heap for the 9.1.2.0 run in case you still find it helpful. Unfortunately, I won't be able to investigate further until next Tuesday... |
So, I got a little bit further:
@headius should I open new tickets for 2 and 3? Or simply rename this one (the OOM is not the problem anymore) |
@MartinKoerner Definitely file one or two new issues depending on how you feel about the other items. Our process is that bugs like this one get closed as "Invalid" so we don't get confusing context. |
This should reduce the overhead of inspecting by avoiding the intermediate StringBuilder/char[] and subsequent conversion to byte[]. There are a few extra objects created along the way (java.lang.String.getBytes, RubyString for class name, etc) but overall memory use should reduce around 3x: a US-ASCII string would need 2x byte size in the StringBuilder, and the builder is still alive when we create the eventual byte[] for a total of 3x max memory use during inspect. Relates to jruby#4127, but does not constitute a fix for memory issues there.
Environment
Provide at least:
Expected Behavior
Actual Behavior
Generator runs for a while, than he hangs on one file (it is the same file every time, but it is not very special in the generated output) and after some time prints:
It will try to get a memory dump...
The text was updated successfully, but these errors were encountered: