-
-
Notifications
You must be signed in to change notification settings - Fork 925
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
jruby-head shoes4 performance regression (on rake spec) #2544
Comments
Can you report back with latest master? |
Updated to latest master. Unfortunately the problems haven't gotten better, the last benchmark has even gotten notably worse from my last 9k snapshot (3.6 seconds --> 6,3 seconds for fill app among others): jruby 1.7.19:
jruby-head:
|
Does the test run spin up any subprocesses? Startup time has increased a bit from 1.7 (we're working on it) and that could add up. The normal benchmark numbers are mostly in line with what I'd expect for current performance, but a reproducible benchmark would help us improve. I'd also be curious about interpreted execution versus JIT versus JIT with invokedynamic on current master. 9k.pre1 is not a good target for benchmarking (at least with JIT enabled), so we'll stick to current master. |
The test run does spin up separate processes, one for every "test suite" - 3 in total (those are the processes I'm aware of). However, the swt specs (one process) are up to 1 minute 10 seconds versus 46 seconds in 1.7. Hm ok, I was a bit alerted as for some cases performance seems to be halved... I could see if I can write a smaller script reproducing the problems but not soon/and no promises (moving this weekend) Same goes for interpreted vs. JIT vs. JIT with invoke dynamic - have to check out the appropriate switches :) Thanks for the feedback Charlie! |
@headius there is some mystery here. His second numbers are with our new faster to get going startup interpreter. So frequent restarts should be much faster than pre1. It sounds like this may not be the case though. This new interpreter is not as fast as old one but it spends a lot less time getting to actual interpretation so I would expect reasonable testing perf. @PragTob I wonder if we broke something in our recent changes since we did nothing which should have affected JIT performance. Your benchmarks slowing down should not have happened. |
It would be good to hear if 9k.pre2 has helped :-) |
Sorry I was sick and just catching up on github notifications :) Sadly results don't seem to have changed much. Time for rake spec in 1.7.19:
Time on jruby-head from just know:
The clear benchmark is also still faster (especially on the first fill): 1.7.19:
head from today:
|
for reference I checked out this commit as it was around the time I opened the issue, so that shoes still works with 1.7: 7c90d16097b87ae61 This is also running JDK 8 now. The difference seems smaller but 1.7 is still faster for the clear benchmark (I gave up getting the tests running again, pulls in new gems that don't play with 1.7 etc.) 1.7:
9.1.6.0:
Still closing out, as probably to hard to reproduce/work out. |
Hello everyone :)
First off, sorry for the relatively coarse grained report. I noticed that with jruby-head/pre1 the performance of shoes4 degraded - at least as mainly shown in running our specs (mainly), so it might as well be related to rspec. Also some very basic benchmarks, see below.
Our
rake spec
runs 3 different test suites. The first one is the shoes-core specs with our mock backend (does nothing), the second one is running our specs with the SWT backend (+ SWT specific specs) and the third one is specs for our packaging. Noteworthy, we do a lot of Java interoperability calls.It is mostly our logic/swt specs that got slower. So for swt it is (as in the example below) 48 seconds on 1.7.19 versus 75 seconds on current jruby-head (built ~2 hours ago). Overall run time is 2minutes 23 seconds versus 3 minutes 33 seconds.
Here is a shortened log of our tests running and the associated time:
full jruby-head test log - warning, lots of gtk warnings in there. Also some jruby deprecation warnings, which I'll work on fixing (already got a shoes ticket)
Also noteworthy, although that travis ci test execution times vary our jruby-head specs are almost reliably 1.5-2.0x slower on travis (as compared to current JRuby) - a travis build
We also have some really basic benchmarks that are a bit slower:
1.7.19:
9k:
As always thank you for all your work! :) I hope to get around to write some more fine grained/not really gui benchmarks that could narrow down and show case the problem better. But no promises, busy times ahead :)
Thanks!
Tobi
The text was updated successfully, but these errors were encountered: