Skip to content

Commit

Permalink
[Truffle] New development readme.
Browse files Browse the repository at this point in the history
  • Loading branch information
@chrisseaton
chrisseaton committed Nov 17, 2016
1 parent c400403 commit 9f5f518
Showing 1 changed file with 161 additions and 60 deletions.
221 changes: 161 additions & 60 deletions truffle/README.md
View file
@@ -1,4 +1,4 @@
# JRuby+Truffle - a High-Performance Truffle Backend for JRuby
# JRuby+Truffle - a High-Performance Implementation of Ruby using Truffle and Graal

The Truffle runtime of JRuby is an experimental implementation of an interpreter
for JRuby using the Truffle AST interpreting framework and the Graal compiler.
Expand All @@ -25,92 +25,193 @@ University Linz](http://ssw.jku.at).
* Lucas Allan Amorim
* Aditya Bhardwaj

And others.

The best way to get in touch with us is to join us in `#jruby` on Freenode, but
you can also Tweet to @chrisgseaton, @nirvdrum, @eregontp or @pitr-ch, or email
chris.seaton@oracle.com.
you can also Tweet to @chrisgseaton, or email chris.seaton@oracle.com.

## User Documentation

The JRuby wiki includes general user documentation for JRuby+Truffle. This
file is documentation for working on the development of JRuby+Truffle.

## Developer Documentation

Normally you want to use the `truffle-head` branch. We merge to `master` once
with every release of GraalVM, so that it is stable when the JRuby classic
team make their releases.

### Requirements

You will need:

* Java 8 (not 9 EA)
* Ruby 2

## Using Truffle
### Developer tool

To run JRuby in Truffle mode, pass the `-X+T` option.
We use a Ruby script to run most commands.

JRuby+Truffle is designed to be run with a JVM that has the Graal compiler. The
easiest way to get this is via the GraalVM, available from the Oracle
Technology Network.
```
$ ruby tool/jt.rb --help
```

https://github.com/jruby/jruby/wiki/Downloading-GraalVM
Most of us create a symlink to this executable somewhere on our `$PATH` so
that we can simply run.

But you can also build it yourself, which you will need to do if you are on the
`truffle-head` branch.
```
$ jt --help
```

https://github.com/jruby/jruby/wiki/Building-Graal
### Building

### What to expect
```
$ jt build
```

JRuby+Truffle is a research project and is not yet a finished product. Arbitrary
programs are very unlikely to run due to missing functionality, and if they do
run they are unlikely to run fast yet due to requiring new functionality to be
tuned. We are at least a year away from being able to run significant programs
without needing new methods to be implemented.
### Testing

Windows is currently not supported.
We have 'specs' which come from the Ruby Spec Suite. These are usually high
quality, small tests, and are our priority at the moment. We also have MRI's
unit tests, which are often very complex and we aren't actively working on now.
Finally, we have tests of our own. The integration tests test more macro use of
Ruby. The ecosystem tests test commands related to Ruby. The gems tests test a
small number of key Ruby 3rd party modules.

## Research
The basic test to run every time you make changes is a subset of specs which
run in reasonable time.

* [Chris Seaton's blog posts](http://www.chrisseaton.com/rubytruffle/)
* C. Seaton. **[Specialising Dynamic Techniques for Implementing the Ruby Programming Language](http://chrisseaton.com/phd/)**. PhD thesis, University of Manchester, 2015.
* M. Grimmer, C. Seaton, R. Schatz, T. Würthinger, H. Mössenböck. **[High-Performance Cross-Language Interoperability in a Multi-Language Runtime](http://chrisseaton.com/rubytruffle/dls15-interop/dls15-interop.pdf)**. In Proceedings of 11th Dynamic Languages Symposium (DLS).
* F. Niephaus, M. Springer, T. Felgentreff, T. Pape, R. Hirschfeld. **[Call-target-specific Method Arguments](https://github.com/HPI-SWA-Lab/TargetSpecific-ICOOOLPS/raw/gh-pages/call_target_specific_method_arguments.pdf)**. In Proceedings of the 10th Implementation, Compilation, Optimization of Object-Oriented Languages, Programs and Systems Workshop (ICOOOLPS), 2015.
* B. Daloze, C. Seaton, D. Bonetta, H. Mössenböck. **[Techniques and Applications for Guest-Language Safepoints](http://chrisseaton.com/rubytruffle/icooolps15-safepoints/safepoints.pdf)**. In Proceedings of the 10th Implementation, Compilation, Optimization of Object-Oriented Languages, Programs and Systems Workshop (ICOOOLPS), 2015.
* M. Grimmer, C. Seaton, T. Würthinger, H. Mössenböck. **[Dynamically Composing Languages in a Modular Way: Supporting C Extensions for Dynamic Languages](http://www.chrisseaton.com/rubytruffle/modularity15/rubyextensions.pdf)**. In Proceedings of the 14th International Conference on Modularity, 2015.
* A. Wöß, C. Wirth, D. Bonetta, C. Seaton, C. Humer, and H. Mössenböck. **[An object storage model for the Truffle language implementation framework](http://www.chrisseaton.com/rubytruffle/pppj14-om/pppj14-om.pdf)**. In Proceedings of the International Conference on Principles and Practices of Programming on the Java Platform (PPPJ), 2014.
* C. Seaton, M. L. Van De Vanter, and M. Haupt. **[Debugging at full speed](http://www.lifl.fr/dyla14/papers/dyla14-3-Debugging_at_Full_Speed.pdf)**. In Proceedings of the 8th Workshop on Dynamic Languages and Applications (DYLA), 2014.
```
$ jt test fast
```

Also see the [Ruby Bibliography](http://rubybib.org), and
[publications specifically on Truffle and Graal](https://wiki.openjdk.java.net/display/Graal/Publications+and+Presentations).
You may also want to regularly run the integration tests.

## Truffle-Specific Functionality
```
$ jt test integration
```

Other tests can be hard to set up and can require other repositories, so we
don't normally run them locally unless we're working on that functionality.

### Running

`jt run` runs JRuby+Truffle. You can use it exactly as you'd run the MRI `ruby`
command. Although it does set a couple of extra options to help you when
developing, such as loading the core lirbary from disk rather than the JAR. `jt
run` prints the real command it's running as it starts.

```
$ ruby ...
$ jt run ...
```

Note that running Ruby without any arguments does not start a shell. You should
run `jt irb` if you want an interactive shell, or `irb` to run the shell of your
system Ruby or a GraalVM tarball.

### Options

There are runtime configuration options that can be set on the command line with
`-Xtrufle.option=value`. To see a list of these run `-Xtruffle...`.
Specify JVM options with `-J-option`.

```
$ jt run -J-Xmx1G test.rb
```

JRuby+Truffle options are set with `-Xtruffle...=...`. For example
`-Xtruffle.exceptions.print_java=true` to print Java exceptions before
translating them to Ruby exceptions.

To see all options run `jt run -Xtruffle...` (literally, with the three dots).

You can also set JVM options in the `JAVA_OPTS` environment variable (don't
prefix with `-J`), or the `JRUBY_OPTS` variable (do prefix with `-J`). Ruby
command line options and arguments can also be set in `JRUBY_OPTS` or `RUBYOPT`
if they aren't JRuby-specific.

### Running with Graal

To run with a GraalVM binary tarball, set the `GRAALVM_BIN` environment variable
and run with the `--graal` option.

```
$ GRAALVM_BIN=../graalvm-0.18-re/bin/java
$ jt run --graal ...
```

You can check this is working by printing the value of `Truffle::Graal.graal?`.

```
$ GRAALVM_BIN=../graalvm-0.18-re/bin/java
$ jt run --graal -e 'p Truffle::Graal.graal?'
```

To run with Graal built from source, set `GRAAL_HOME`.

```
$ GRAAL_HOME=../../graal/graal-core-workspace/graal-core
$ jt run --graal ...
```

Set Graal options as any other JVM option.

```
$ jt run --graal -J-Dgraal.TraceTruffleCompilation=true ...
```

We have flags in `jt` to set some options, such as `--trace` for
`-J-Dgraal.TraceTruffleCompilation=true` and `--igv` for
`-J-Dgraal.Dump=Truffle`.

### Testing with Graal

The basic test for Graal is to run our compiler tests. This includes tests that
things partially evaluate as we expect, that things optimise as we'd expect,
that on-stack-replacement works and so on.

```
$ ... jt test compiler
```

### Benchmarking with Graal

### Truffle Module
Checkout the `all-ruby-benchmarks` and `benchmark-interface` repositories above
your checkout of JRuby. We usually run like this.

The `Truffle`, `Truffle::Graal`, `Truffle::Debug` and `Truffle::Interop` modules include
Truffle-specific functionality. They aren't documented yet, so you'll need to read the
source to know what you can do.
```
$ jt benchmark ../all-ruby-benchmarks/classic/mandelbrot.rb --simple
```

### Debugger
Output is iterations per second, printed roughly every second (more frequently
for the first few iterations).

`Truffle::Debug.break` will enter a shell and allow to introspect the program.
THe best way to set JVM options here is to use `JAVA_OPTS`.

If you don't want to modify the program to include a call to
`Truffle::Debug.break` you can break the main thread externally. Run with the
instrumentation server enabled, `-Xtruffle.instrumentation_server_port=8080`.
Then you can send a message to the runtime to break at the current location:
### Benchmarking without Graal

curl http://localhost:8080/break
You can turn off Graal if you want using `--no-graal`.

### Stack Server
```
$ jt benchmark --no-graal ../all-ruby-benchmarks/classic/mandelbrot.rb --simple
```

To dump the call stacks of a running Ruby program in Truffle, run with the
instrumentation server enabled and the `passalot` option,
`-Xtruffle.instrumentation_server_port=8080 -Xtruffle.passalot=1`. Then you can
dump the current call stack of all threads:
You can benchmark JRuby Classic using `-Xclassic` in `JRUBY_OPTS`.

curl http://localhost:8080/stacks
```
$ JRUBY_OPTS=-Xclassic jt benchmark ../all-ruby-benchmarks/classic/mandelbrot.rb --simple
```

## Workflow Tool
You can benchmark an entirely different implementation using the
`JT_BENCHMARK_RUBY` environment variable.

Truffle is built as part of JRuby, but if you are working on the Truffle code
base you may appreciate the `jt` tool. To use it alias it in your shell
`function jt { ruby tool/jt.rb "$@"; }`, then run `jt --help` to see the commands
available. You need a standard `ruby` from your system to run the tool.
```
$ JT_BENCHMARK_RUBY=ruby jt benchmark ../all-ruby-benchmarks/classic/mandelbrot.rb --simple
```

For example:
### mx and integrating with other Graal projects

* `jt build` builds JRuby and Truffle
* `jt run args...` runs JRuby in Truffle mode
* `jt run --graal args...` runs JRuby in Truffle mode, using Graal
* `jt test fast` runs a subset of Truffle tests
JRuby can also be built and run using `mx`, like the other Graal projects. This
is intended for special cases such as integrating with other Graal projects, and
we wouldn't recommend using it for normal development. If you do use it, you
should clean before using `jt` again as having built it with `mx` will change
some behaviour.

0 comments on commit 9f5f518

Please sign in to comment.