Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Truffle Setup issues with jt.rb and #exec #4346

Closed
davetrollope opened this issue Nov 30, 2016 · 5 comments
Closed

Truffle Setup issues with jt.rb and #exec #4346

davetrollope opened this issue Nov 30, 2016 · 5 comments
Milestone

Comments

@davetrollope
Copy link

davetrollope commented Nov 30, 2016

Environment

ch-nb-647:jruby dtrollope$ /Users/dtrollope/graaltest/jruby/bin/jruby --version
jruby 9.1.7.0-SNAPSHOT (2.3.1) 2016-11-28 6f2e683 Java HotSpot(TM) 64-Bit Server VM 25.71-b01-internal-jvmci-0.22 on 1.8.0_92-b14 +jit [darwin-x86_64]
ch-nb-647:jruby dtrollope$ /Users/dtrollope/graalvm-0.18-re/bin/java -version
java version "1.8.0_92"
Java(TM) SE Runtime Environment (build 1.8.0_92-b14)
Java HotSpot(TM) 64-Bit Server VM (build 25.71-b01-internal-jvmci-0.22, mixed mode)
ch-nb-647:jruby dtrollope$ type -a ruby
ruby is /Users/dtrollope/graalvm-0.18-re/bin/ruby
ruby is /usr/bin/ruby
ch-nb-647:jruby dtrollope$ /Users/dtrollope/graalvm-0.18-re/bin/ruby --version
jruby+truffle 9.1.6.0-SNAPSHOT (2.3.1) 2016-11-02 8a74a73 Java HotSpot(TM) 64-Bit Server VM 25.71-b01-internal-jvmci-0.22 on 1.8.0_92-b14 [darwin-x86_64]
ch-nb-647:jruby dtrollope$ /usr/bin/ruby --version
ruby 2.0.0p648 (2015-12-16 revision 53162) [universal.x86_64-darwin15]
ch-nb-647:jruby dtrollope$ uname -a
Darwin ch-nb-647 15.6.0 Darwin Kernel Version 15.6.0: Thu Sep  1 15:01:16 PDT 2016; root:xnu-3248.60.11~2/RELEASE_X86_64 x86_64

Expected Behavior

I was running through the dev setup on https://github.com/jruby/jruby/tree/truffle-head/truffle and ran in to a couple of issues.

Running jt ruby --graal -e 'p Truffle::Graal.graal?' should return true

Actual Behavior

When I run jt ruby --graal -e 'p Truffle::Graal.graal?' I get a No such file or directory (Errno::ENOENT). I supplemented jt.rb to dump what was being passed to exec() and verified the paths being used are valid, see output below:

ch-nb-647:jruby dtrollope$ jt ruby --graal -e 'p Truffle::Graal.graal?'
$ JAVACMD=/Users/dtrollope/graalvm-0.18-re/bin/java /Users/dtrollope/graaltest/jruby/bin/jruby -X+T -Xtruffle.core.load_path=/Users/dtrollope/graaltest/jruby/truffle/src/main/ruby -e 'p Truffle::Graal.graal?'
ARG [{"JAVACMD"=>"/Users/dtrollope/graalvm-0.18-re/bin/java"}, "/Users/dtrollope/graaltest/jruby/bin/jruby", "-X+T", "-Xtruffle.core.load_path=/Users/dtrollope/graaltest/jruby/truffle/src/main/ruby", "-e", "p Truffle::Graal.graal?", {}]
ARG {"JAVACMD"=>"/Users/dtrollope/graalvm-0.18-re/bin/java"}
Error during command: ruby --graal -e p Truffle::Graal.graal?
/Users/dtrollope/graaltest/jruby/tool/jt.rb:285:in `exec': No such file or directory (Errno::ENOENT)
    from /Users/dtrollope/graaltest/jruby/tool/jt.rb:285:in `raw_sh'
    from /Users/dtrollope/graaltest/jruby/tool/jt.rb:683:in `run'
    from /Users/dtrollope/graaltest/jruby/tool/jt.rb:688:in `ruby'
    from /Users/dtrollope/graaltest/jruby/tool/jt.rb:1356:in `send'
    from /Users/dtrollope/graaltest/jruby/tool/jt.rb:1356:in `main'
    from /Users/dtrollope/graaltest/jruby/tool/jt.rb:1364:in `<main>'
ch-nb-647:jruby dtrollope$ ls -ld /Users/dtrollope/graalvm-0.18-re/bin/java /Users/dtrollope/graaltest/jruby/bin/jruby /Users/dtrollope/graaltest/jruby/truffle/src/main/ruby
-rwxr-xr-x  1 dtrollope  1132860720  12571 Nov 28 08:32 /Users/dtrollope/graaltest/jruby/bin/jruby
drwxr-xr-x  4 dtrollope  1132860720    136 Nov 25 22:27 /Users/dtrollope/graaltest/jruby/truffle/src/main/ruby
-rwxr-xr-x@ 1 dtrollope  1132860720   3520 Nov  2 14:42 /Users/dtrollope/graalvm-0.18-re/bin/java

Now - the curious thing is if I remove the hash with the environment setup from what is passed to #exec, it all works:

ch-nb-647:jruby dtrollope$ git diff /Users/dtrollope/graaltest/jruby/tool/jt.rb
diff --git a/tool/jt.rb b/tool/jt.rb
index 6fc3da1..6813bb0 100755
--- a/tool/jt.rb
+++ b/tool/jt.rb
@@ -279,6 +279,9 @@ module ShellUtils
     end

     if use_exec
+args.shift
+puts "ARG #{args}"
+puts "ARG #{args.first}"
       result = exec(*args)
     elsif timeout
       result = system_timeout(timeout, *args)
ch-nb-647:jruby dtrollope$ jt ruby --graal -e 'p Truffle::Graal.graal?'
$ JAVACMD=/Users/dtrollope/graalvm-0.18-re/bin/java /Users/dtrollope/graaltest/jruby/bin/jruby -X+T -Xtruffle.core.load_path=/Users/dtrollope/graaltest/jruby/truffle/src/main/ruby -e 'p Truffle::Graal.graal?'
ARG ["/Users/dtrollope/graaltest/jruby/bin/jruby", "-X+T", "-Xtruffle.core.load_path=/Users/dtrollope/graaltest/jruby/truffle/src/main/ruby", "-e", "p Truffle::Graal.graal?", {}]
ARG /Users/dtrollope/graaltest/jruby/bin/jruby
true

I'm not familiar with how #exec is processing the environment hash, but something odd is happening. Currently I am relying on java being in the path.

@nirvdrum
Copy link
Contributor

Hmm . . . I'm sorry you've run into problems. Can you please let me know the commit SHA for the version of JRuby you have checked out? I tried to deduce it from your output above, but I think your checked out version is different than the JRuby 9.1.6.0-SNAPSHOT you're using. I'm having a bit of difficulty matching up your backtrace as a result.

@davetrollope-uptake
Copy link

ch-nb-647:jruby dtrollope$ git rev-parse HEAD
2eab42d
ch-nb-647:jruby dtrollope$ git status
On branch truffle-head
Your branch is up-to-date with 'origin/truffle-head'.
Changes not staged for commit:
(use "git add ..." to update what will be committed)
(use "git checkout -- ..." to discard changes in working directory)

modified:   tool/jt.rb

no changes added to commit (use "git add" and/or "git commit -a")

@eregon eregon added the truffle label Nov 30, 2016
@eregon eregon added this to the truffle-dev milestone Nov 30, 2016
@eregon
Copy link
Member

eregon commented Nov 30, 2016

@davetrollope In your patch, you first do args.shift, which will drop the first argument.
Could you print all args, by adding p args at the beginning raw_sh and post the output?
I think that would help to debug :)

From the mailing lost:

To close the loop here, yes I freshly cloned the repo - but just for grins I pulled and reran. Same results.
I think you were asking about the system ruby? Here are my installed versions for completeness:

ch-nb-647:jruby dtrollope$ type -a ruby
ruby is /Users/dtrollope/graalvm-0.18-re/bin/ruby
ruby is /usr/bin/ruby
ch-nb-647:jruby dtrollope$ /Users/dtrollope/graalvm-0.18-re/bin/ruby --version
jruby+truffle 9.1.6.0-SNAPSHOT (2.3.1) 2016-11-02 8a74a73 Java HotSpot(TM) 64-Bit Server VM 25.71-b01-internal-jvmci-0.22 on 1.8.0_92-b14 [darwin-x86_64]
ch-nb-647:jruby dtrollope$ /usr/bin/ruby --version
ruby 2.0.0p648 (2015-12-16 revision 53162) [universal.x86_64-darwin15]

This is the problem, the fact that graalvm-0.18-re/bin is in $PATH.
It means that the ruby that is used to run tool/jt.rb is graalvm-0.18-re/bin/ruby (that is, JRuby+Truffle from GraalVM) but we rather expect MRI to run that script.
I can reproduce this with:

PATH=../graalvm-0.18-re/bin:$PATH tool/jt.rb ruby --graal -e 'p Truffle::Graal.graal?'

For now, a simple fix is to not have graalvm-0.18-re/bin in your $PATH.
I'll add a guard to jt to warn about this case. It also means we don't support exec fully yet with these arguments.
But in any case, running jt with JRuby+Truffle itself is suboptimal: it pays the startup cost twice, so it's best to keep MRI in PATH for faster startup.

Can you confirm removing graalvm-0.18-re/bin from your $PATH solves the problem?
Sorry for the issue and thanks for the report!

eregon added a commit that referenced this issue Nov 30, 2016
…Truffle.

* It's not a good recipe for fast startup.
* See #4346.
@davetrollope-uptake
Copy link

davetrollope-uptake commented Dec 1, 2016

Sure - I tossed the shift in above my debugs - didn't make sense after.. Here you go:

ch-nb-647:jruby dtrollope$ jt ruby --graal -e 'p Truffle::Graal.graal?'
ARG1 [{"JAVACMD"=>"/Users/dtrollope/graalvm-0.18-re/bin/java"}, "/Users/dtrollope/graaltest/jruby/bin/jruby", "-X+T", "-Xtruffle.core.load_path=/Users/dtrollope/graaltest/jruby/truffle/src/main/ruby", "-e", "p Truffle::Graal.graal?", {:use_exec=>true}]
$ JAVACMD=/Users/dtrollope/graalvm-0.18-re/bin/java /Users/dtrollope/graaltest/jruby/bin/jruby -X+T -Xtruffle.core.load_path=/Users/dtrollope/graaltest/jruby/truffle/src/main/ruby -e 'p Truffle::Graal.graal?'
true
ch-nb-647:jruby dtrollope$ git diff
diff --git a/tool/jt.rb b/tool/jt.rb
index 3c08502..41e48e9 100755
--- a/tool/jt.rb
+++ b/tool/jt.rb
@@ -269,6 +269,7 @@ module ShellUtils
   end

   def raw_sh(*args)
+puts "ARG1 #{args}"
     options = args.last.is_a?(Hash) ? args.last : {}
     continue_on_failure = options.delete :continue_on_failure
     use_exec = options.delete :use_exec
@@ -280,6 +281,7 @@ module ShellUtils
     end

     if use_exec
+      args.shift # DST
       result = exec(*args)
     elsif timeout
       result = system_timeout(timeout, *args)

You are right - using the system ruby without the graal version in the path works:

ch-nb-647:jruby dtrollope$ jt ruby --graal -e 'p Truffle::Graal.graal?'
$ JAVACMD=/Users/dtrollope/graalvm-0.18-re/bin/java /Users/dtrollope/graaltest/jruby/bin/jruby -X+T -Xtruffle.core.load_path=/Users/dtrollope/graaltest/jruby/truffle/src/main/ruby -e 'p Truffle::Graal.graal?'
true
ch-nb-647:jruby dtrollope$ type -a ruby
ruby is /usr/bin/ruby
ch-nb-647:jruby dtrollope$ set | grep GRAAL
GRAALVM_BIN=/Users/dtrollope/graalvm-0.18-re/bin/java

Thanks for the speedy responses! Your comments about the startup cost are interesting. This has been one of my biggest issues with jruby and rake tasks - the startup costs often outweigh the trivial nature of rake tasks. That is of course a subject for another thread but I would certainly encourage improvements in that area.

Glad we at least identified a specific issue with #exec that can be addressed too.

Thanks again!
Dave

@eregon
Copy link
Member

eregon commented Dec 1, 2016

You are welcome!
We are working on startup time in various ways (different VM, serializing the AST of core Ruby files, etc), but of course there is nothing like not paying the startup cost twice 😃
I'll close this and open an issue for the unsupported exec form.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants