JRuby starts executing Enumerator code too soon #4583
Comments
|
The problem is that the thread started to power Enumerator#next does not pause after yielding the nil result to the yielder. It doesn't pause until the next iteration, because there's nobody waiting for that value. As a result, it executes past the yield. I believe I already fixed this for Fiber. What I didn't do is reimplement Enumerator#next in terms of Fiber, as MRI does. |
|
If you are really just using this yielder pattern, you could swap in Fiber and it would work ok. fiber = Fiber.new {
Fiber.yield nil
puts "EXECUTED"
}
fiber.resumeThis will not output anything, because you only resume and get the first value, after which the Fiber properly transfers control back and waits. |
|
More on reimplementing Enumerator#next in terms of Fiber... I have prototyped this before, and the amount of code required is very small (all implemented in Ruby). The piece that's missing is our short-circuited logic for when the data being enumerated comes from an Array. RubyEnumerator, implemented in Java, has at least two kinds of If (when?) we move Enumerator#next to use a fiber, we will still want to keep the optimized versions for known collections. |
|
Thanks you @headius! |
As described in jruby/jruby#4583, JRuby starts executing code after the first `yield` even though we requested only the first element, resulting in the first chunk being overriden with the second chunk before it was even returned. We work around this by not using a buffer string, therefore each retrieved chunk is a new string, so even if JRuby immediately retrieves the second chunk, it won't affect the first chunk.
Using Enumerator interface like Enumerator#next wouldn't work correctly on JRuby with IOs, because JRuby eagerly loads the next element before it returns the current, and because each next chunk overwrites the previous one, it is impossible to retrieve first chunk via Enumerable#next. jruby/jruby#4583 In order to avoid potential bugs in the future of someone wanting to use this Enumerator interface, we remove this entirely and mandate people to pass in a block.
Using Enumerator interface like Enumerator#next wouldn't work correctly on JRuby with IOs, because JRuby eagerly loads the next element before it returns the current, and because each next chunk overwrites the previous one, it is impossible to retrieve first chunk via Enumerable#next. jruby/jruby#4583 In order to avoid potential bugs in the future of someone wanting to use this Enumerator interface, we remove this entirely and mandate people to pass in a block.
|
I'm closing this one as a duplicate of #5007. |
Environment
Expected Behavior
For the given code
MRI 2.4.1 doesn't output anything. I expected JRuby to behave the same, to pause and not to execute any code after
yield, since we requested only the first element of the Enumerator.Actual Behavior
However, JRuby outputs
In other words, JRuby started executing the code after
yield, even though we only asked for the first element.The text was updated successfully, but these errors were encountered: