You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
On JRuby 9k, that script will OOM with the default 512MB heap. Under JRuby 1.7.20, I can run it with only a 16MB heap without issues. This script basically parses a string and creates an array of arrays with the parsed data. You end up with something like:
When looking at a heap dump, those "Translation X" strings are RubyStrings backed by unique ByteLists where each ByteList has a byte[] array containing the entirety of remaining str and then an offset and length pointing to just a tiny portion of the byte array.
So, it's like the ByteLists think they're being shared here but they actually are not being shared, ending up with multiple copies of very large ByteLists. They either need to be shared or not shared but pruned to not contain the excess string data.
The text was updated successfully, but these errors were encountered:
This fixesjruby#3019, where in certain cases it's possible to end up with
RubyString instances backed by very large unique ByteLists where only
a tiny portion of the bytes in the ByteList are actually needed.
What was happening is `modify` was being called on RubyString
instances too eagerly, resulting in unnecessary duplication of their
underlying ByteList instances instead of sharing when possible.
The two changes here are in `RubyString#newFrozen` and
`RubyString#resize` and I believe are more correct based on the C
implementation of these methods.
For `RubyString#newFrozen`, which roughly corresponds to
`rb_str_new_frozen` in
C (https://github.com/ruby/ruby/blob/v2_2_2/string.c#L932), the logic
looks to me to return the shared string and I don't see it calling
`str_make_independent`, which is roughly the same thing as
`RubyString#modify`. So, I just removed the call to `modify`.
For `RubyString#resize`, which roughly corresponds to to
`rb_str_resize` in C, shared strings are only made independent if
their length
differ (https://github.com/ruby/ruby/blob/v2_2_2/string.c#L2155). Thus,
instead of unconditionally calling `modify` here, I moved it into the
conditions that are true when the length differs.
After working through this problem a bit in IRC, we've come up with a fairly simple reproduction script here:
https://gist.github.com/bbrowning/90c2296e048fb806b4a1
On JRuby 9k, that script will OOM with the default 512MB heap. Under JRuby 1.7.20, I can run it with only a 16MB heap without issues. This script basically parses a string and creates an array of arrays with the parsed data. You end up with something like:
When looking at a heap dump, those "Translation X" strings are RubyStrings backed by unique ByteLists where each ByteList has a byte[] array containing the entirety of remaining
str
and then an offset and length pointing to just a tiny portion of the byte array.So, it's like the ByteLists think they're being shared here but they actually are not being shared, ending up with multiple copies of very large ByteLists. They either need to be shared or not shared but pruned to not contain the excess string data.
The text was updated successfully, but these errors were encountered: