Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Set/Array #include? performance degradation #3389

Closed
abrandoned opened this issue Oct 12, 2015 · 5 comments
Closed

Set/Array #include? performance degradation #3389

abrandoned opened this issue Oct 12, 2015 · 5 comments
Labels
ir performance
Milestone
Invalid or Duplicate

Comments

@abrandoned
Copy link

Between Jruby-1.7.20.1 and JRuby-9.0.1.0 there seems to be a significant degradation in the performance of Array and Set #include? methods along with other Enumerable methods (select/reject) that we can't seem to come up with a plausible explanation as to why (as we have looked through the code)

It looks as if the performance has been cut in half with respect to these methods

JRuby-1.7.20.1 (Array#include?)

require 'benchmark'
a = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
Benchmark.measure do
  1_000_000.times do
    a.include?(8)
  end
end
# <Benchmark::Tms:0x307262ee @stime=0.0, @total=0.1899999999999995, 
# @real=0.13199996948242188, @cutime=0.0, @utime=0.1899999999999995, @label="", @cstime=0.0> 
Benchmark.measure do
  Benchmark.measure do
    10_000_000.times do
      a.include?(8)
    end
  end
end
#<Benchmark::Tms:0x2cb2e8f0 @stime=0.0, @total=1.1900000000000004, 
#@real=0.9170000553131104, @cutime=0.0, @utime=1.1900000000000004, @label="", @cstime=0.0>

JRuby-9.0.1.0 (Array#include?)

#<Benchmark::Tms:0x4166d6d3 @stime=0.0, @total=0.47000000000000064, 
# @real=0.29648134601302445, @cutime=0.0, @utime=0.47000000000000064, @label="", 
# @cstime=0.0> 
...
#<Benchmark::Tms:0x1604506f @stime=0.010000000000000009, @total=2.5199999999999996, 
#@real=2.3672206070041284, @cutime=0.0, @utime=2.51, @label="", @cstime=0.0>

JRuby-1.7.20.1 (Set#include?)

s = Set.new(a)
Benchmark.measure do
  10_000_000.times do
    s.include?(8)
  end
end
#<Benchmark::Tms:0x3a29efbf @stime=0.009999999999999953, @total=1.1500000000000006, 
# @real=1.0849997997283936, @cutime=0.0, @utime=1.1400000000000006, @label="", 
# @cstime=0.0>

JRuby-9.0.1.0 (Set#include?)

#<Benchmark::Tms:0x21c88db @stime=0.0, @total=2.66, @real=2.620377297978848, @cutime=0.0, 
# @utime=2.66, @label="", @cstime=0.0>
@headius
Copy link
Member

headius commented Oct 12, 2015

Worth a gander.

@enebo
Copy link
Member

enebo commented Oct 12, 2015

I do not think this is likely a slowdown in Array#nclude? (I did not check Set) but in IR and its processing of blocks:

a = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

require "benchmark/ips"

Benchmark.ips do |x|
  x.config(:time => 10, :warmup => 5)

  x.report("Array#include?") do
    a.include?(8)
    a.include?(8)
    a.include?(8)
    a.include?(8)
    a.include?(8)
    a.include?(8)
    a.include?(8)
    a.include?(8)
    a.include?(8)
    a.include?(8)
  end

  x.report("times do") do
    1_000_000.times do
      a.include?(8)
    end
  end

  x.report("while do") do
    i = 0
    while i < 1_000_000
      a.include?(8)
      i += 1
    end
  end

end

for 1.7:

../jruby-1_7/bin/jruby ../snippets/include1.rb 
Calculating -------------------------------------
      Array#include?   105.570k i/100ms
            times do     1.000  i/100ms
            while do     2.000  i/100ms
-------------------------------------------------
      Array#include?      2.856M (± 5.4%) i/s -     28.504M
            times do     18.088  (± 5.5%) i/s -    181.000 
            while do     25.515  (± 3.9%) i/s -    256.000

For 9k:

jruby ../snippets/include1.rb 
Calculating -------------------------------------
      Array#include?    76.132k i/100ms
            times do     1.000  i/100ms
            while do     2.000  i/100ms
-------------------------------------------------
      Array#include?      2.703M (± 9.4%) i/s -     26.722M
            times do     13.683  (± 7.3%) i/s -    137.000 
            while do     25.728  (± 3.9%) i/s -    258.000

In completely unrolled loop we are <10% difference with 1.7 winning. With 'times' we are significantly slower on 9k so there is something we are not opting well in empty loop. For a simple while you can see 9k actually runs things faster. So I suspect neither Array nor Set methods are to blame but our block overhead.

@film42 film42 mentioned this issue Oct 12, 2015
Closed
@headius
Copy link
Member

headius commented Oct 12, 2015

My numbers for Integer#times were better than @enebo's and only about 10% off from 1.7. This is a known regression in overall block dispatch performance that we have plans to fix.

On my system, Array#include? also has nearly identical performance between 1.7 and 9k, so I don't think there's anything to fix for this bug. Block perf is a known issue and will be improved in future releases of 9k.

@headius headius closed this as completed Oct 12, 2015
@enebo
Copy link
Member

enebo commented Oct 12, 2015

I finally updated to u60 and I see the same perf difference as @headius now.

@abrandoned
Copy link
Author

upgrading to u60 also resolved this perf diff on my machine, thanks!

@enebo enebo added this to the Invalid or Duplicate milestone Oct 13, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
ir performance
Projects
None yet
Milestone
Invalid or Duplicate
Development

No branches or pull requests
3 participants
@headius @enebo @abrandoned