We've tested this out and let it run until destruction. The regexp cache fills up the heap and the transaction time increases and increases (90 secs instead of 1-2 secs). Is there any way to limit the size of this cache?

same thing happened in my environment.

repro code

# example.rb
count = 0
while(true)
  count += 1
  Regexp.new("test#{count}")
end

c:\workspace\sandbox\reg-cache>jruby -v
jruby 1.7.16.1 (1.9.3p392) 2014-10-28 4e93f31 on Java HotSpot(TM) Client VM 1.7.0_17-b02 +jit [Windows 7-x86]

c:\workspace\sandbox\reg-cache>jruby -J-verbose:gc -J-XX:+HeapDumpOnOutOfMemoryError example.rb
[GC 4412K->1804K(15872K), 0.0141832 secs]
[GC 6220K->2974K(15872K), 0.0181254 secs]
[GC 7390K->4579K(15872K), 0.0201395 secs]
[GC 8995K->5779K(15872K), 0.0158137 secs]
[GC 10195K->6735K(15872K), 0.0126515 secs]
...
[Full GC 494975K->494974K(494976K), 0.7089635 secs]
[Full GC 494975K->494975K(494976K), 0.8630243 secs]
[Full GC 494975K->494975K(494976K), 0.9114314 secs]
[Full GC 494975K->494975K(494976K), 0.9192651 secs]
[Full GC 494975K->494975K(494976K), 0.8271494 secs]
java.lang.OutOfMemoryError: Java heap space
Dumping heap to java_pid7940.hprof ...
Heap dump file created [526303819 bytes in 8.361 secs]
[Full GC 494975K->6176K(494976K), 0.1836955 secs]
[Full GC 64846K->6187K(494976K), 0.0662034 secs]
Error: Your application used more memory than the safety cap of 500M.
Specify -J-Xmx####m to increase it (#### = cap size in MB).
Specify -w for full OutOfMemoryError stack trace

We have found the source of the problem, it was slightly more hidden than your reproduction. We were doing a substitution (when trying to filter params) that happened on every transaction.

config.filter_parameters += known_sensitive_fields << lambda { |key, value|
      if value.present? && API::ExpectedFields.request_keys.exclude?(key) then
        value.sub!(value, "[FILTERED]") unless API::ExpectedFields.is_dynamic?(key)
      end
    }

The value.sub! in there was the problem, the value was changing with every request and being added to the Regex Cache. We fixed the problem by using value.replace("[FILTERED]") instead - no more leak.

I do think that the cache entries should perhaps have a TTL or a maximum size to avoid something like this.

I can't reproduce this on 1.7.16.1 or master - the SoftReferences get collected reliably.

Using:

count = 0
while(true)
  count += 1
  Regexp.new("test#{count}" * 500000)
end

Master:

1.7.16.1:

$ java -version
java version "1.7.0_45"
OpenJDK Runtime Environment (fedora-2.4.3.0.fc18-x86_64 u45-b15)
OpenJDK 64-Bit Server VM (build 24.45-b08, mixed mode)

@cheald
Thank you for confirmation.

It seems GC collect except system to access to RegexpCache at high frequency.

• "access to RegexpCache at high frequency": ex) when interval time to make Regexp object is shorter than -XX:SoftRefLRUPolicyMSPerMB value

I think, reason to not reproduce at cheald's code is interval time to make Regexp object becomes longer at cheald' code.

In actuary, "access to RegexpCache at high frequency" does not occur nearly.
So, no problem at my system.

thank you.

Thank you for reporting this. Yes, this implementation is insufficient. We are exploring another.

Thanks guys - really glad that our 2 weeks of pain has at least resulted in JRuby being improved. Thanks for your hard work!

Should be fixed now in jruby-1_7 and master. The new cache appears to run @cheald's benchmark 3x faster than MRI, 2x faster than Rubinius, and retained heap holds steady.

Not sure how best to write a test for this. Will try to add one into the JUnit suite.