Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How about Porting the Contents of org.jruby.bench to JMH? #4752

Closed
original-brownbear opened this issue Aug 27, 2017 · 7 comments
Closed

How about Porting the Contents of org.jruby.bench to JMH? #4752

original-brownbear opened this issue Aug 27, 2017 · 7 comments

Comments

@original-brownbear
Copy link
Contributor

There are some pure Java micro-benchmarks in org.jruby.bench, that simply use public static void main( invocation and don't seem to have any clean warmup logic and such.
I would suggest to maybe port those to JMH cleanly and once done would also be very happy to contribute further benchmark cases.
I have quite a few performance spots in Logstash I'm working on right now that I'd love to optimize by contributions to JRuby and having JMH as a tool available would be great for speeding this kind of work up.

I'll probably create some JMH code around JRuby, but if there's interest in a PR I'll do a cleaner job => just let me know if you think it's something you'd like to have :)

@kares
Copy link
Member

kares commented Aug 28, 2017

personally, wanted to have JMH available as a test dependency a few times already :)
but than I usually just went with micro benchmarking Ruby directly. not sure if all are worth converting, maybe a few - some new benchmarks might be interesting (e.g. not really sure about the intern-ing you did)

@original-brownbear
Copy link
Contributor Author

@kares nice :)

some new benchmarks might be interesting (e.g. not really sure about the intern-ing you did)

Exactly! This would make demonstrating/evaluating improvements much easier :)

That said, let's do it! (this should be fairly quick and trivial:)) Where should we put the Maven module for this? Just top level jmh-bench/pom.xml maybe?

@kares
Copy link
Member

kares commented Aug 28, 2017

not sure if we would need a new 'module' ... maybe some of the existing places can be (re-)used :

  1. have JMH available as a test dependency on core and write bms under org.jruby.bench
    ... although I am not sure if that would make much sense -> they won't be run during tests anyway
  2. bench directory contains .rb (micro) benchmarks - that might be converted to a maven module
    ... having src/main/java for JMH and src/main/ruby for Ruby-style benchmarks

JRuby usually uses pom.rb but that shouldn't matter a raw pom.xml will be just fine, since it will be an isolated module anyway. might be some build tricks needed to keep it up-to-date - could be solved later.

@original-brownbear
Copy link
Contributor Author

@kares let's do 2. :) JMH tests should be an isolated module that depends on whatever module you want to benchmark ideally.
Quote from http://openjdk.java.net/projects/code-tools/jmh/

The recommended way to run a JMH benchmark is to use Maven to setup a standalone project that depends on the jar files of your application. This approach is preferred to ensure that the benchmarks are correctly initialized and produce reliable results. It is possible to run benchmarks from within an existing project, and even from within an IDE, however setup is more complex and the results are less reliable.

This is how I set it up for Logstash https://github.com/elastic/logstash/tree/master/logstash-core/benchmarks and I think we could do the same by putting some pom.rb under /bench right? Could even add a comfortable Maven build target for it like I did in LS for rebuild + running those JMH benchmarks in a single step :)

original-brownbear added a commit to original-brownbear/jruby that referenced this issue Aug 28, 2017
@kares kares added this to the Non-Release milestone Aug 31, 2017
kares pushed a commit that referenced this issue Aug 31, 2017
First step towards JMH benchmarks as discussed in #4752
@original-brownbear
Copy link
Contributor Author

@kares port the other two benchmarks in that package to JMH as well? :)

@kares
Copy link
Member

kares commented Sep 3, 2017

@original-brownbear if you're into that I do not see why not ... should be useful as an inspiration at least

@original-brownbear
Copy link
Contributor Author

@kares sure it was a really small effort left anyways :) + looks so much nicer now imo.
Check out #4774 if you have a sec :)

@kares kares closed this as completed in 222fda5 Sep 4, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants