Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Truffle -> RVM #3768

Closed
ylluminate opened this issue Mar 30, 2016 · 32 comments
Closed

Truffle -> RVM #3768

ylluminate opened this issue Mar 30, 2016 · 32 comments
Milestone

Comments

@ylluminate
Copy link

ylluminate commented Mar 30, 2016

With the attention that @chrisseaton's interview has gotten, it seems high time to figure out how to get this going in RVM for higher accessibility and playing from developers.

Has anyone looked into how to pull this together for RVM as of yet?

@chrisseaton
Copy link
Contributor

Any standard distribution of JRuby 9000 includes Truffle. Truffle is moving fast, so jruby-head is always best for anyone experimenting. The only problem is that at the moment the JVM doesn't include the modifications needed to run it efficiently. OpenJDK 9 will solve that in the future.

I'm not an RVM user myself, so I need to check this works seamlessly, but the general idea at the moment is that you can download the GraalVM from Oracle (https://github.com/jruby/jruby/wiki/Downloading-GraalVM) and then run with something like:

$ JAVACMD=GraalVM-0.10/jre/bin/javao bin/jruby -X+T ...

So you get JRuby as usual, you just need a different VM.

There are some other options like building Graal from scratch (https://github.com/jruby/jruby/wiki/Building-Graal), but these are not normally required.

We should do some things like set up tests using chruby, rbenv and RVM in CI to make sure this always works.

In the past we did have a tarball that included Graal and JRuby, so you got the whole thing transparently, but we've moved away from that because we want to provide JS and R in the same package, and those are closed source and require click-through licenses so we can't ship in a tarball.

Let me know if that doesn't help you do what you wanted to do.

If you are experimenting with Truffle, I'd recommending talking to us in IRC so we can manage expectations and solve any problems quickly.

@ylluminate
Copy link
Author

Thanks, will play with this in the near future. I'm thinking that getting it into RVM (and rbenv, etc.) will really help get things moving in the right direction as far as getting more eyes on it and using it for gem dev, etc. Seems like the right time to push this forward.

FYI, might want to look into using Gitter with the Sameroom IRC conduit that will create a nice cohesive env for support / chat. I've found myself moving more and more away from IRC to Gitter with Opal and such and it seems that Sameroom is really friendly at helping open source projects as such in this area.

@chrisseaton
Copy link
Contributor

We do have Gitter https://gitter.im/jruby/jruby, but it isn't integrated with IRC at the moment.

@chrisseaton chrisseaton added this to the truffle-dev milestone Apr 2, 2016
@ylluminate
Copy link
Author

@chrisseaton with tweets like this, we're going to see some real exodus action if we don't start putting forth some interesting results from Truffle. :)

I realize it's easy to say things like that, and there's no criticism intended, just wanted to share the thought since too often marketing is the last thing we think about as developers.

@chrisseaton
Copy link
Contributor

This issue will be resolved when JDK 9 EA supports Graal, which should be soon. Then we will tell people to just use the normal JRuby releases and JDK 9 EA. We'll be making sure that works and documenting it as soon as we can. Or people can use GraalVM from OTN today https://github.com/jruby/jruby/wiki/Downloading-GraalVM, but this isn't automatable in RVM because it has a click-through licence.

More generally, we're working very hard and fast on completeness. We are actively running and testing increasingly large parts of Rails and the rest of the Ruby ecosystem, and we're making leaps in areas such as C extensions. I think we already employ more people working on Ruby than anyone else anywhere in the world and the team is growing. So we're making as much progress as we can 👍

@ylluminate
Copy link
Author

ylluminate commented May 22, 2016

That's wonderful to hear @chrisseaton and this month's TIOBE Index most certainly bodes well for all of us: http://www.tiobe.com/tiobe_index

May Headline: Ruby equals best position ever

Ruby is currently at position 8 in the TIOBE index. This is equal to the highest position it reached in December 2008. This second peak is quite the opposite of the first one. In 2006 the Ruby evangelists were shouting the language to the top. There was no room for self reflection or discussion: Ruby and its programming environment Rails were about to conquer the world. And they succeeded in this to some extent. Ruby became the language for fancy start up companies with their tight time to market schedules. It was even awarded "Programming Language of 2006" until its popularity peaked at the end of 2008. Then scalability appeared to be a problem. Performance dropped significantly if much data needed to be processed. In the beginning of 2010 one of those fancy start ups, Twitter, started to replace Ruby by Java for this same reason. The Ruby evangelists vanished to other cool new languages and Ruby seemed to be on its way out. But not really. After being out the limelight for quite some years, Ruby's popularity started to grow again at the beginning of 2015. This time it seems to be a more sustainable growth based on actual usage of the language instead of shouting.

@ylluminate
Copy link
Author

ylluminate commented May 26, 2016

@chrisseaton again, it's interesting to see the momentum and serious desire and effort. People are really willing to learn something new to obtain this performance, so we can only imagine the reception we're going to see from Truffle!: http://www.mikeperham.com/2016/05/25/sidekiq-for-crystal/

Why Crystal?
I wanted to use a language that was a good complement to Ruby. Its syntax is similar enough to Ruby that I can reuse a lot of code but it adds a huge leap in performance. In summary:

  • very similiar syntax to Ruby
  • 3-5x faster than Ruby 2.3 on most code
  • 3x smaller in memory footprint
  • statically typed
  • compiles to a single, 1MB binary! Deployment is easy.
  • comes with a large, useful standard library
  • In other words, Ruby is friendly, flexible and works for most usecases while Crystal is fast and efficient for those usecases where performance is paramount. Use each where appropriate.

How productive is it? I started this project from scratch not knowing the language at all a week ago and had the core job processor running in 3 days.

@headius
Copy link
Member

headius commented May 27, 2016

FWIW, JRuby's already 20% faster than MRI, and that's with us running pure-Ruby hiredis versus MRI using an extension. If the extension were fixed to load on JRuby 9k we could be close to the Crystal numbers already.

@ylluminate
Copy link
Author

ylluminate commented May 30, 2016

Thanks for noting that @headius. If I i had some time I'd (try to) run his test against Truffle.

@ylluminate
Copy link
Author

@chrisseaton what's the scoop now from your perspective? Been a bit tied up in other projects and wanted to perhaps regroup on this. Clearly OpenJDK 9 is starting to wind down towards release from the scheduling I've seen. Seems like time might be about right to start increasing exposure. I heard that Truffle was passing all specs as of a couple months ago as per your presentation up on YouTube.

@chrisseaton
Copy link
Contributor

I'm waiting for just one more piece of the puzzle, which is an OpenJDK 9 EA build with JVMCI (the interface that Graal uses) that is fully functional. As soon as that happens I will revisit this.

@ylluminate
Copy link
Author

Gotcha. Where can we find the status of that? Who's working on adding that, etc?

@chrisseaton
Copy link
Contributor

It looks like open JVMCI issues for Java 9 are here https://bugs.openjdk.java.net/browse/JDK-8165755?jql=labels%20%3D%20jvmci.

However I didn't mention another problem with using JVMCI - it won't actually include Graal, so you'd need to get that from somewhere. Maybe it could be on Maven, but then how would it be installed and found.

I should answer this question from scratch - the recommended way to use JRuby+Truffle if you're already a JRuby user is to install JRuby as normal, separately install GraalVM (http://www.oracle.com/technetwork/oracle-labs/program-languages/overview/index.html) and then use it via JAVACMD=.

@ylluminate
Copy link
Author

ylluminate commented Nov 12, 2016

Seeing JVMCI mentioned a good bit in http://download.java.net/java/jdk9/changes/jdk-9+143.html -- what's the story there at the moment?

UPDATE: Note the following: https://github.com/graalvm/graal-core/issues/220

@chrisseaton
Copy link
Contributor

It does now work. You can use it with the jt command in the JRuby source repository.

$ JVMCI_BIN=../../graal/jdk-9-ea+143_osx-x64_bin/Contents/Home/bin/java JVMCI_GRAAL_HOME=../../graal/graal-core-workspace-9/graal-core/ jt run --graal -e "puts Truffle::Graal.graal?"

That's equivalent to running this:

JAVACMD=../../graal/jdk-9-ea+143_osx-x64_bin/Contents/Home/bin/java jruby/bin/jruby -X+T -J-d64 -J-XX:+UnlockExperimentalVMOptions -J-XX:+EnableJVMCI -J--add-exports=java.base/jdk.internal.module=com.oracle.graal.graal_core -J--module-path=../../graal/graal-core-workspace-9/graal-core/../truffle/mxbuild/modules/com.oracle.truffle.truffle_api.jar:../../graal/graal-core-workspace-9/graal-core/mxbuild/modules/com.oracle.graal.graal_core.jar --no-bootclasspath -e 'puts Truffle::Graal.graal?'

So that's still a little complicated. And there's no place to get built versions of those jars at the moment (and they also need to be built with Java 9). If you had a VM with an EA build installed though, and had your own version of these jars, it would now work.

@ylluminate
Copy link
Author

I still wonder about wrapping this into RVM such that downloading Java 9 and setting it up in a proper isolated environment just available to the JRuby / RVM might do the trick of simplifying this to get people to really start using and testing as initially noted... Even the license issue could just have a prompt in RVM for triggering acceptance and download.

@chrisseaton
Copy link
Contributor

The only problem is the Graal and Truffle jars. But the source code of those is GPL+classpath, so someone could build and publish them if they wanted to. I'm afraid we're not planning to do that at Oracle at the moment though.

@ylluminate
Copy link
Author

What's the holdup on the Oracle front wrt that? BTW, have fun in Cincinnati? :) Was two hours away, but didn't have a chance to make it over... :(

@chrisseaton
Copy link
Contributor

It's just not part of our strategy at this stage to be making Graal releases on Maven. Can you even make Java 9 releases on Maven? What would happen if they changed the class file format before release? I think it would end up as a mess. The GraalVM (http://www.oracle.com/technetwork/oracle-labs/program-languages/overview/index.html) is what we're doing for distribution and we want to focus on that at the moment.

@ylluminate
Copy link
Author

Hmmm. I just hate to see such a "high" or, perhaps, not-very-marketable entry point for Truffle testing. The easier it is to install, the more folks will start testing and raving about it (e.g. rvm install jruby-truffle is might attractive vs other ways...)

@ylluminate
Copy link
Author

@chrisseaton really appreciated your Ruby Conf presentation. Your final synopsis was fantastic and you summed things up nicely. Might consider throwing up a running status summarizing your final points and where to find the actual tickets that would track status of these items such as OpenSSL, Nokogiri, various DB drivers, etc. Obviously still heavily pro figuring out RVM, but after your presentation I think we've all got a much more clear understanding of what's going on and where this is going. Extremely excited.

@chrisseaton
Copy link
Contributor

@ylluminate thanks - actually we don't have any public issues visible for those major tasks. I'll open some, thanks.

@ylluminate
Copy link
Author

Great, thanks @chrisseaton! Where should I look for these tasks? Here on the jruby issues or elsewhere?

@chrisseaton
Copy link
Contributor

I haven't done it yet but it's in my to-do. I'll be doing it as part of my end-of-year wrap up email and plan for 2017.

@ylluminate
Copy link
Author

Sounds good. I know a lot of folks really chomping at the bit and still feel jumping into testing / dev a bit daunting atm. :)

@ylluminate
Copy link
Author

@chrisseaton how're things going? Any news on the previous points?

@chrisseaton
Copy link
Contributor

Which points, sorry? We moved this issue here oracle/truffleruby#9 and I should have closed this one. Were there other points you wanted to bring up again?

@ylluminate
Copy link
Author

Thanks @chrisseaton. Back on 12/16 you mentioned that you were going to open public issues for those major tasks from Ruby Conf.

@chrisseaton
Copy link
Contributor

The three user-facing issues that are most important at the moment are

This is mostly what is holding up people being able to experiment more and run more gems.

@ylluminate
Copy link
Author

Thanks very much for spearheading this and updating us as to those key points @chrisseaton! Everyone I talk to is absolutely itching for this easy accessibility via rvm, etc. right now, always with the additional remark that this is really going to increase testing and adoption.

If the clickthrough must be maintained, then some simple instructions, shell level, will ease the burden for "higher level" users (frankly at this point I think dealing with these additional steps is more than acceptable):

1) Go to <URL>,
2) Accept the license,
3) Download the package,
4) Place package in <SOME_PATH; perhaps ./>
5) Press <ENTER> to continue installation!

@chrisseaton
Copy link
Contributor

If the clickthrough must be maintained

We don't think it needs to be, but it may take a little while longer to sort this out.

We'll think about your suggestion. You don't think it would be annoying to see TruffleRuby as available in RVM and then be told to go off somewhere else to actually get it?

@ylluminate
Copy link
Author

Well.... Not really, I don't think that "annoying" is the word for it at this phase. There is such an impetus in users' minds right now just to get their hands on it, that I think it's an acceptable step at this moment. Clearly at some point it will need to be gone, but you're talking about some really excited folks here who have a little more capacity to accept such an intermediate step towards the ultimate goal of rvm install truby.

The lip smacking right now is audible... 😸

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants