"org.jruby.runtime.scopes.DynamicScope1 only supports scopes with 1 variables" with Deeply Nested Blocks #4739
Comments
|
Update: Outright forcing global sync on |
|
Looking into this. |
|
I was unable to reproduce using JRuby master and your scripts. Switching to 9.1.13 branch now. |
|
@headius I ran jruby -J-Djruby.jit.logging=true -J-Djruby.jit.threshold=0 -J-Djruby.jit.logging.verbose=true -Xcompile.invokedynamic=true reproducer.rb > /dev/nulland I can still reproduce it with current |
|
I was unable to get it to fail with OracleJDK 8 but when I switched to a Zulu build of Java 7 I got the DynamicScope error right away. I'm guessing this may be a problem with concurrent classloading in Java 7 that was fixed later on. Working on a patch. |
|
@headius not so sure about that, for me it fails reliably on ➜ $ java -version
java version "1.8.0_131"
Java(TM) SE Runtime Environment (build 1.8.0_131-b11)
Java HotSpot(TM) 64-Bit Server VM (build 25.131-b11, mixed mode) |
|
It's very intermittent but I've managed to get both failures to appear. The DynamicScope error I suspect is a concurrent classloading problem. The stack overflow is more likely a race condition in IR. It appears to be getting mixed up which block it should be passing to a call and ending up looping back on itself. |
|
This is a bugger to hunt down but I'm proceeding from your workaround of synchronizing all of block JIT. |
|
A few other findings:
|
This effectively means the background JIT will usually be limited to one thread operating at a time. How much this will slow down or bottleneck the JIT is not clear but this fix will let us know if we need a deeper look at synchronization of the JIT and IR structures modified at JIT time. Part of work for #4739.
|
I've pushed a few commits that seem to have helped things. They're on the jit-conc-fixes branch for you to test out.
Give that branch a try and let us know how it looks. |
|
@headius as I said in the linked LS issue, this fix looks very promising, so far I couldn't reproduce the reported problems with it :) |
|
Fixes merged to master and 9.1.13.0. |
|
@headius thanks a lot! Any ETA on the |
|
We should release 9.1.13 this week or next. |
|
@headius we just hit this issue on JRuby 9.2.12.0. Previously we were on JRuby 9.2.9.0 and it didn't occur. |
|
Running Java VM OpenJDK 64-Bit Server VM Java Version 1.8.0_252 JRuby Version 9.2.12.0 JRuby Revision db01a49 |
EDIT:
I was able to create a non-Logstash reproducer by creating a somewhat arbitrary set of classes with a bunch of blocks and multi-threading.
See reproduced error here:
https://github.com/original-brownbear/jruby-jit-issue-reproducer#jit-dynamicscope-size-error (code is in that repo, link points at the error reproduced (top of the readme contains a second error that I reproduced using this code, it seems we're randomly going into an endless loop here, but I haven't 100% verified that it's not just stupidity on my end introducing recursion of some kind) the the command used to trigger it )
In JRuby
9.1.12.0we can still reproduce #4516 with the latest Logstash versions in a case of many nested blocks and a number of threads likely trying to JIT them at the same time.The exception I see is:
the last few steps here are:
using:
Sorry for not having a shorter reproducer, but as far as I understand the JRuby codebase and the explainations in #4516 and related issue this probably very hard to reproduce in isolation.
FORCE) compilation makes the issue go away (or at least so unlikely that I couldn't reproduce it so far)I'll try to get you more details to work with.
The text was updated successfully, but these errors were encountered: