-
-
Notifications
You must be signed in to change notification settings - Fork 925
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Coverage levels discrepancies between JRuby 1.7.x and JRuby 9.1.x #4819
Comments
Nope. We just have a bug here. JRuby 9k uses completely different internals than JRuby 1.7.x. JRuby 9k now converts a syntax tree into a set of virtual machine instrs and emits a line_num instr. In this case I am guessing the instrance variable + if ends up putting a line number instr in a place where we think we can cull it (or never emit it in the first place). |
@enebo To add to this, there are other examples of this failed coverage, not just for 'if else' with instance variables. I saw a similar issue with .to_json() not getting covered. It's a more general problem I think, not limited to this example. |
@mullermp yeah it would not surprise me very much. In fact it probably has nothing to do with the if/else specifically and more to do with us emitting line_num in combination of two elements (like if and an instance variable). I have fixed multiple coverage issues already so either this is a regression or oddly a case with no coverage in our tests. |
Ok looked into this and wowsers it was not in IR at all. It was back in the AST. Basically anything which represents an ident during lexing ivar, cvar, gvar, keywords, constants would ask lexer for current position but lexer will advance past the ident in cases where the ident is last thing on a line and advance to next line (not 100% of the time but most of the time). Fortunately, the lexer has a special position value for idents called tokline. I switched all obvious nodes made directly from idents to use tokline and it seems to have fixed this issue and probably many others. I also ran simplecov specs and while there were 4 errors....none were related to positions changing. Thinking about whether we have any nice tests for this somewhere else too. Obviously this is not being captured in any test runs we normally do. |
…uby 9.1.x Basically anything which represents an ident during lexing ivar, cvar, gvar, keywords, constants would ask lexer for current position but lexer will advance past the ident in cases where the ident is last thing on a line and advance to next line (not 100% of the time but most of the time). Fortunately, the lexer has a special position value for idents called tokline. I switched all obvious nodes made directly from idents to use tokline and it seems to have fixed this issue and probably many others.
Awesome. I'll be trying this after release and I'll re-open if there's still any issues with coverage. |
@mullermp yeah fantastic. We want perfect compat for coverage and will prioritize if we can. |
@zendes can you open this up as a new issue please? |
I think this is part of the same issue. The original example I posted were just a simple one. I have issues such as coverage missing in multi-line hashes. And multi-line if statements conditions that are missed but can be resolved by editing the condition. e.g. foo << bar unless bool1 || #HIT foo << bar unless bool1 || #HIT |
After upgrading my Rails app's JRuby version from JRuby 1.7.27 to JRuby 9.1.13 I noticed a drop in coverage levels in my code base from 100% to roughly 94%.
I'm using the SimpleCov Gem to output the rspec coverage but it uses Ruby's underlying Coverage module to get the actual coverage numbers.
(rspec 3.6 and simplecov 0.15)
Environment
JRuby 9.1.13
rspec 3.6 and simplecov 0.15
Rails
Expected Behavior
Behavior prior to upgrade

Behavior in ruby 2.4.1 using (rbenv)

Actual Behavior
Behavior post upgrade to JRuby 9.1.13

Here's a screen shot of a happy test to prove that testing does indeed work in my JRuby 9.1.13 setup (removed the instance variable)

As seen in the screen shots of the simple test above, coverage is incorrectly skipped in jruby9000 in certain cases (instance variable in a boolean condition in the above case)
I can post more examples if necessary for debugging.
Am I just missing some special flag when calling rspec, or is there something bigger going on here?
I've run with --dev, --debug, and -Xcli.debug=true
The text was updated successfully, but these errors were encountered: