-
-
Notifications
You must be signed in to change notification settings - Fork 925
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Performance regression with json/pure
from 1.7 to 9k
#3921
Comments
Link with #4112. |
Ok, this is still slower for me on 9.2, but the difference is closer: 1.7.27 jit: 38s Note this is just the warmup run. I'm going to bump to 9.2.1 to go along with other perf work post 9.2. |
I think we finally caught up in upcoming 9.2.1.0:
|
I don't understand though why my runs are so short... |
Oh but I should say we are markedly faster with these short runs than 9.2 and 9.1 so something definitely got better. Will try and lengthen this out and see how it does with a longer execution. |
HAHAHA...I just realized I ran timeparse bench and not the json one. So good news the time parse work we did for 9.2.1.0 shows us much better...now to actually re-bench with the proper benchmark. |
JIT has passed 1.7.x now but interp is still like 40% slower. Ratio has not changed much since this was reported. This should stay open as json/pure probably a good test case for something expensive in the interp. |
Updated numbers for JRuby 9.2.8 versus JRuby 1.7.27 on my machine.
The numbers for json/pure are perhaps slightly improved from what @enebo found them in September, with JIT comfortably faster (30s vs 36s for tail) and interpreter still around 1.75x the time of the interpreter (86s vs 50s). I'm going to take a bit of time and see if there's a reason why the interpreter is so much slower. |
A quick allocation profile shows one possible avenue: the interpreted mode allocates significantly more Object[] than the jitted mode. Most of these appear to be coming from the interpreter's temp variables, which must be allocated anew for each call to any interpreted method that has local or temporary variables (which basically means all of them, since it's a register machine). @enebo Maybe it's worth doing another experiment with using a large per-thread Object[] and carving off sections of it rather than allocating every time? I don't see anything obvious in the sampled profile; the overhead is mostly within joni in both cases. |
Frustratingly, another attempt at pooling Object[] does not appear to have helped. I tried with and without value-clearing (we need to clear to avoid anchoring references beyond the life of a method call) and it still always ends up roughly the same level of perf. Here's the patch for what it's worth: https://gist.github.com/a7eed3fc8f9870bc2c0ff6b41e892c15 |
I made a few more small attempts to improve interpreter performance, but nothing really made a difference:
FWIW I also measured TruffleRuby against json/pure, to get a complete picture of how we all do. I am not sure if there's a way to disable TruffleRuby JIT, so this is their JIT performance with the native image:
|
De-targetting. I do have an idea on how to improve cases and branching/jumping in the interpreter (load/stores to temp array can be elided in single 1:1 use:def cases to a local variable) and this will have some effect...but...the work to implement that is going to be substantial (all calls/lookups must be in same local variable scope as all branches). There are bigger fish to fry so we will leave this open and it may be a funday friday acitivity. |
Environment
JRuby 1.7.20.1, JRuby 9.1.0.0, JRuby 9.1.1.0; run from Java via
ScriptingContainer
, with various values tested for thesetCompileMode
method. Standalone script execution with no frameworks. Benchmarked JSON gemsjson/pure
,json/ext
, andjrjackson
.Expected Behavior
Expected performance for 9.1.x to be on par with performance for 1.7.x.
Actual Behavior
For
json/pure
, JRuby 9k performance is markedly slower (75%) whencompileMode
is set toOFF
.I have set up a github repo with a simple reproducer here:
https://github.com/cprice404/jruby9k-benchmarks/tree/19a2719e4edddc17e3f1fabbf311b9d80c6eba80/json-comparison
I also graphed some of the results in a google spreadsheet here:
https://docs.google.com/spreadsheets/d/1M64QXK_RSFd05_Te9PamTv7ALi7XlAkhWeyRoAoIFZU/edit#gid=866554277
Here are some relevant numbers from the benchmark output (the rest of it is available in the linked reproducer repro if you are interested):
The other JSON libraries besides
json/pure
are much, much faster, and are in the ballpark of JRuby 1.7 performance. Also, when usingjson/pure
with any othercompileMode
besidesOFF
, the delta between JRuby 1.7 and JRuby 9k drops significantly.Unfortunately, our application codebase currently vendors a fork of
json/pure
that will make it very difficult for us to switch to a different JSON backend.Also, we haven't had great luck in our experiments with using
compileMode
JIT
, most likely because it seems to be very expensive during the warmup of aScriptingContainer
, and we destroy and create newScriptingContainer
s with some frequency during our application execution.I recognize that many users will be able to avoid using
json/pure
, and thus the priority of solving this may be diminished; however, my hope is that perhaps the reproducer highlights some kind of memory usage or performance issue that would be relevant to other JRuby applications, and thus, that looking into optimizations for this scenario might yield improvements in other areas as well. Or, at least, that it would be interesting to try to characterize why the performance changes so drastically between JRuby 1.7 and 9k.The text was updated successfully, but these errors were encountered: