Sorry, see I misspelled , should be 'require "test2".

You'll probably need to pass -Xaot.loadClasses=true to JRuby or -Djruby.aot.loadClasses=true to the JVM. (see @headius comment - #3018 (comment))

HTH

Thank you very much for pointing this out.
However this option seems not to be supported in JRUBY-COMPLETE:

JRUBY lists aot.loadClasses as a valid option:

jruby -v
jruby 9.0.0.0 (2.2.2) 2015-07-21 e10ec96 Java HotSpot(TM) Client VM 24.75-b04 on 1.7.0_75-b13 +jit [Windows 7-x86]
jruby -Xproperties

Look for .class before .rb to load AOT-compiled code

Options: [true, false], Default: false.

#aot.loadClasses=false

JRUBY-COMPLETE ( based on the same version of JRUBY) does not seem to support this option:

java -jar jruby-complete-9.0.0.0.jar -v
jruby 9.0.0.0 (2.2.2) 2015-07-21 e10ec96 Java HotSpot(TM) Client VM 24.75-b04 on 1.7.0_75-b13 +jit [Windows 7-x86]

java -jar jruby-complete-9.0.0.0.jar -Xaot.loadClasses=true
jruby: invalid extended option aot.loadClasses=true (-X will list valid options)

java -jar lib\java\jruby-complete-9.0.0.0.jar -X
jruby: missing argument
Extended options:
-X-O run with ObjectSpace disabled (default; improves performance)
-X+O run with ObjectSpace enabled (reduces performance)
-X-C disable all compilation
-X-CIR disable all compilation and use IR runtime
-X+C force compilation of all scripts before they are run (except eval)
-X+CIR force compilation and use IR runtime
-X+JIR JIT compilation and use IR runtime
-X+T use Truffle
-Xsubstring? list options that contain substring in their name
-Xprefix... list options that are prefixed wtih prefix

Further progress:
The second option you mentioned, -Djruby.aot.loadClasses=true to the JVM does work with JRUBY-COMPLETE. It exposed however a new problem that require_relative does not work with compiled classes. After replacing
require_relative "filename"
with
require File.expand_path('filename', File.dirname(__FILE__))
all the rest seems to work fine.
There was an earlier issue concerning require_relative, but that was cured before this release. This current problem seems to only occur for compiled programs.
CZ

@CeesZ yes jruby-complete can not use those -X options since they are meant for the launcher which does convert them to -Djruby. ones for the java invocation.

Yes, thank you for explaining; I understand that now.

@CeesZ We'll narrow this to just the require_relative issue. Can you provide a small sample repository that shows this problem? With the jruby.aot.loadClasss property set, I don't see why require_relative wouldn't work just fine.

Sorry for late reply. At the moment on holiday in Scotland with only occasional access to internet. Will be back home at the end of this week and work on a small sample repository.
=Cees

On 4 sep. 2015, at 01:21, Charles Oliver Nutter notifications@github.com wrote:

@CeesZ We'll narrow this to just the require_relative issue. Can you provide a small sample repository that shows this problem? With the jruby.aot.loadClasss property set, I don't see why require_relative wouldn't work just fine.

—
Reply to this email directly or view it on GitHub.

I constructed the following archive (test1.jar) for testing require_relative:

/app
  |
  |__ test1.rb                          puts "test1: __FILE__ = " + __FILE__
  |                                     require_relative 'test2'
  |
  |__ test2.rb                          puts "test2: __FILE__ = " + __FILE__
  |                                     require_relative 'sub/test3'
  |
  |__/sub
       |__ test3.rb                     puts "test3: __FILE__ = " + __FILE__
       |                                require_relative 'test4'
       |
       |__ test4.rb                     puts "test4: __FILE__ = " + __FILE__

run with jruby-complete:

java -Djruby.aot.loadClasses=true -cp test1.jar;jruby-complete-9.0.0.0.jar org.jruby.Main -e "require 'uri:classloader:/app/test1'"

output (as expected)

test1: __FILE__ = uri:classloader:/app/test1.rb
test2: __FILE__ = uri:classloader:/app/test2.rb
test3: __FILE__ = uri:classloader:/app/sub/test3.rb
test4: __FILE__ = uri:classloader:/app/sub/test4.rb

test2.jar is identical to test1.jar, with .rb files replaced by their .class files; using same java command:

java -Djruby.aot.loadClasses=true -cp test2.jar;jruby-complete-9.0.0.0.jar org.jruby.Main -e "require 'uri:classloader:/app/test1'"

instead of loading test2.class from uri:classloader:/app/test2.class it attempts
to find test1.rb outside the jar probaly using the working directory:

test1: __FILE__ = uri:classloader:/app/test1.class
Errno::ENOENT: No such file or directory - C:\Projects\test_project\app\test1.rb
          realpath at org/jruby/RubyFile.java:805
  require_relative at uri:classloader:/jruby/kernel/kernel.rb:21
             <top> at app/test1.rb:2
           require at org/jruby/RubyKernel.java:940
            (root) at uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/core_ext/kernel_require.rb:1
             <top> at -e:1

I also noticed some odd behavior of __FILE__ when using .class files,
maybe this gives a clue as to what is going wrong:
for the same instance of __FILE__

(supposedly a String object)

puts __FILE__          # => uri:classloader:/app/test1.class
puts "#{__FILE__}"     # => app/test1.rb
puts __FILE__.class    # => String

when using .rb files:

puts __FILE__          # => uri:classloader:/app/test1.rb
puts "#{__FILE__}"     # => uri:classloader:/app/test1.rb

Hope this will be of some help to solve the problem.

==Cees

Sorry tried to attach the test jars, but does not seem to work

I can reproduce it on my machine (not windows) and get

Errno::ENOENT: No such file or directory - /Users/cmeier/projects/active/jruby/app/test1.rb
          realpath at org/jruby/RubyFile.java:805
  require_relative at uri:classloader:/jruby/kernel/kernel.rb:11
             <top> at app//test1.rb:2
           require at org/jruby/RubyKernel.java:939
            (root) at uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/core_ext/kernel_require.rb:1
             <top> at -e:1

it does not complain not finding test2 but not finding test1 on the filesystem - indeed this is looking odd.

@CeesZ thanks for the testcase - my own trial never failed ;)

OK, here are the test jars:

https://www.dropbox.com/s/fe1n7p8juuj532b/test1.jar?dl=0
https://www.dropbox.com/s/d5kscle3z9lzxju/test2.jar?dl=0

@mkristian maybe it is caused by a File.dirname request on "#{__FILE__}" instead of __FILE__ ?

@mkristian Do you have any thoughts on this? We'd like to put 9.0.2 out soon.

looking now . . .

the caller in this line
https://github.com/jruby/jruby/blob/master/core/src/main/ruby/jruby/kernel/kernel.rb#L7
is running the jar with the ruby scripts

uri:classloader:/app/test1.rb:2:in `<top>'
/Users/cmeier/projects/active/jruby/lib/ruby/stdlib/rubygems/core_ext/kernel_require.rb:1:in `require'
/Users/cmeier/projects/active/jruby/lib/ruby/stdlib/rubygems/core_ext/kernel_require.rb:1:in `(root)'
-e:1:in `<top>'

with the compiled files it is

app//test1.rb:2:in `<top>'
/Users/cmeier/projects/active/jruby/lib/ruby/stdlib/rubygems/core_ext/kernel_require.rb:1:in `require'
/Users/cmeier/projects/active/jruby/lib/ruby/stdlib/rubygems/core_ext/kernel_require.rb:1:in `(root)'
-e:1:in `<top>'

adjusting the file from the kernel require_relative method with file = "uri:classloader://#{file.sub(/.rb/, '.class')}" fixes things.

@headius stepping into the caller is new territory for nothing I am able to do today

the class has __FILE__ compiled into the class, in the above example it is app//test1.rb which is different from what the FileResource from the LoadService delivers: uri:classloader://app/test1.class

this is probably also the reason why there are two representation of FILE around, one from the class-file and one from the LoadService.

@mkristian @headius This is the last blocker for 9.0.2.0. There multiple things I do not understand at play here :)

FILE in a compiled version of a .rb file SHOULD be a .rb name shouldn't it? I mean compilation is just a different way of storing a .rb. In that sense I think it is weird to ever display this as .class. So I do not think the solution is to do rename FILE. My stance is compilation is just a different save format for a .rb file and it should not change any user-visible behavior from the act of compilation.

If I require_relative a AOT compiled class should I even care what the extension is of the file I am in? I do care about the base parent path but I would think we would do that correctly? Can't we just strip the extension off the caller data and this will just work? We don't actually know if we will be loading a .rb or a .class from this file.

@enebo I thought absolute path does need the file extension. but just tried it on both jruby + mri and it works. yes, this is one thing which can be improved easily.

not sure if this is related or not, but there are more inconsistencies:
test.rb

puts __FILE__
puts "#{__FILE__}"

then compile and put it into a jar

jrubyc test.rb
jar -cf some.jar test.class
rm test.rb

now

$ jruby -e 'require "uri:classloader:/test.class"'
uri:classloader:/test.class
uri:classloader:/test.class

some with "." on the classpath

$ java -cp .:jruby-complete-9.0.2.0-SNAPSHOT.jar org.jruby.Main -e 'require "uri:classloader:/test.class"'
uri:classloader:/test.class
uri:classloader:/test.class

but

$ java -cp some.jar:jruby-complete-9.0.2.0-SNAPSHOT.jar org.jruby.Main -e 'require "uri:classloader:/test.class"'
LoadError: no such file to load -- uri:classloader:/test.class
  require at org/jruby/RubyKernel.java:939
  require at uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/core_ext/kernel_require.rb:54
    <top> at -e:1

and whatever jruby or jruby-complete they all can not find the file without ".class"

$ jruby -e 'require "uri:classloader:/test"'
LoadError: no such file to load -- uri:classloader:/test
  require at org/jruby/RubyKernel.java:939
  require at /Users/cmeier/projects/active/jruby/lib/ruby/stdlib/rubygems/core_ext/kernel_require.rb:54
    <top> at -e:1

@mkristian could this be because of special logic for uri:classloader somehow?

@enebo I really would like to take this uri:classloader: path out of the equation and once the following example works, it should basically fix the original problem here:

test.rb

p __FILE__
Dir.chdir('/tmp')
require_relative 'a'

and a.rb with

puts 'hello'

then execute it with MRI

$ ruby -v test.rb
ruby 2.2.3p173 (2015-08-18 revision 51636) [x86_64-darwin14]
"test.rb"
hello

and with jruby

$ ../bin/jruby -v test.rb
jruby 9.0.2.0-SNAPSHOT (2.2.2) 2015-10-14 d0bcfee Java HotSpot(TM) 64-Bit Server VM 25.51-b03 on 1.8.0_51-b16 +jit [darwin-x86_64]
"test.rb"
Errno::ENOENT: No such file or directory - /private/tmp/t.rb
          realpath at org/jruby/RubyFile.java:814
  require_relative at uri:classloader:/jruby/kernel/kernel.rb:11
             <top> at t.rb:3

slightly different setup but same error.

diff --git a/core/src/main/ruby/jruby/kernel/kernel.rb b/core/src/main/ruby/jruby/kernel/kernel.rb
index b4dd5e13..6ec4164 100644
--- a/core/src/main/ruby/jruby/kernel/kernel.rb
+++ b/core/src/main/ruby/jruby/kernel/kernel.rb
@@ -8,7 +8,12 @@ module Kernel
     file = $` # just the filename
     raise LoadError, "cannot infer basepath" if /\A\((.*)\)/ =~ file # eval etc.

-    absolute_feature = File.expand_path(relative_arg, File.dirname(File.realpath(file)))
+    dir = File.dirname(file)
+    if File.exists?(dir)
+      absolute_feature = File.expand_path(relative_arg, File.dirname(File.realpath(file)))
+    else
+      absolute_feature = File.join(dir, relative_arg.sub(/\\.[^.\/]+$/, ''))
+    end

     require absolute_feature
   end

will "fix" this original example but it will fail in case there is an empty directory app where you execute the application. then it will fail in the same manner as before.

searching some files or directories on the filesystem based on the PWD is not the right thing to do.

I pushed a change that makes AOT-compiled IR set a proper FILE based on how it was loaded. My approach was to make the Filename operand retrieve the filename from its containing scope, which delegates up to whatever the top scope is in that context. I'm not sure if this is the best way but it was simple to implement.

I know this is changing behavior but I find it strange that FILE is transformed with AOT'd to be .class. It is the file it is loaded from but I think it means an AOT-version of a ruby file now has different outward appearange. It also means it is impossible to run debugging tools on AOT'd code. Can anyone comment on what would happen if we make FILE report original .rb instead of its ultmate .class?

After talking to @headius I think my desire for this change is unimportant for this particular fix. If we are running debugging tools and AOT those instructions into the compiled class it will emit the actual name so FILE is less clear how it is important (or not). Something about it still bugs me but I will open another issue when/if I can think of a better reason for disliking this .rb/.class mismatch on AOT'd file.

so my suspicion on File.realpath bit me here: #3401

anyways can we close this one now ?

@mkristian If there's no other changes needed for require_relative and friends, we can close this. FILE behavior should match 1.7 now.