[Q] dir variable in jar file. #4611

hiroyuki-sato · 2017-05-17T16:20:05Z

I would like to ask about __dir__ variable in jar archive.

1. environment

JRuby: 9.1.7.0
OS: macOS 10.12.4
java: 1.8.0_112

2. expected and actual behavior

2.1 Expected Behavior

As the following test result, I think __dir__ should return A or B

A: uri:classloader:/gems/liquid-4.0.0/lib
B: /path/to/jar_file.jar!/gems/liquid-4.0.0/lib

2.2 Actual Behavior

The __dir__ returns strange path like /path/cwd:classloader:/gems/liquid-4.0.0/lib.
/path/cwd is the current working directory which I execute java command.

3. Question

Is this JRuby issue or not?

4. Details

This sample project make a single jar file with liquid-4.0.0 gem package.
The liquid package are locate below.

jar tvf ./build/libs/jruby__dir_test-jruby.jar | grep liquid.rb
  3386 Wed May 17 23:56:38 JST 2017 gems/liquid-4.0.0/lib/liquid.rb

When I put the following codes in gems/liquid-4.0.0/lib/liquid.rb
The result were below.

p "                 File.dirname(__FILE__): #{File.dirname(__FILE__)}"
p "File.expand_path(File.dirname(__FILE__): #{File.expand_path(File.dirname(__FILE__))}"
p "                                __dir__: #{__dir__}"

program	result
`File.dirname(__FILE__)`	uri:classloader:/gems/liquid-4.0.0/lib
`File.expand_path(File.dirname(__FILE__)`	uri:classloader:/gems/liquid-4.0.0/lib
`__dir__`	/private/tmp/jruby__dir_test/uri:classloader:/gems/liquid-4.0.0/lib

(/private/tmp/jruby__dir_test is the java execution directory)

And I also test Dir.exist? command.

Dir.exist?	result	note
uri:classloader:/gems/liquid-4.0.0/lib	true
/path/to/jar_file.jar!/gems/liquid-4.0.0/lib	true
/path/to/jar_file.jar!uri:classloader:/gems/liquid-4.0.0/lib	false
/path/to/project_dir/uri:classloader:/gems/liquid-4.0.0/lib	false	`__dir__` result

5. Test codes.

cd /tmp
git clone https://github.com/hiroyuki-sato/jruby__dir_test
cd jruby__dir_test

./gradlew clean
git checkout -b liquid_4_0_0 liquid_4_0_0
./gradlew jrubyJar
java -jar build/libs/jruby__dir_test-jruby.jar

this java code execute the following Ruby code.

puts JRUBY_VERSION
puts RUBY_VERSION

require 'liquid'
require 'pp'
pp Liquid::Continue

The result is the below.

9.1.7.0
2.3.1
NameError: uninitialized constant Liquid::Continue
Did you mean?  Continuation
  const_missing at org/jruby/RubyModule.java:3344
         <main> at classpath:jar-bootstrap.rb:6

pp Liquid::Continue raise an exception because
lib/liquid.rb in liquid-4.0.0 use the following code and __dir__ return strange path.

Dir["#{__dir__}/liquid/tags/*.rb"].each { |f| require f }

Best regards.

The text was updated successfully, but these errors were encountered:

enebo · 2017-05-17T18:02:01Z

@hiroyuki-sato this is definitely a bug. We should be returing similarly to the FILE.

hiroyuki-sato · 2017-05-18T01:42:56Z

Hello, @enebo
Thank you for quick replying.
Please let me know if you need more information about this issue.

Best regards.

kares · 2017-06-08T19:43:04Z

@hiroyuki-sato since the plugin packs in jruby ... any chance you could force it to use latest JRuby 9.1.10?

kares · 2017-06-08T20:09:38Z

I see now, this simply never worked (the __dir__ version) since JRuby has always being doing a kind of :
new File(__FILE__).getAbsolutePath() in its dir impl ... changing that might have consequences.
basically the notion or uri:classloader: is "special-care" on places such as dirname, missing in __dir__

headius · 2017-06-08T20:16:52Z

@kares I think we need the same special-case logic here, or better, make this call hopefully-existing logic that does the right thing already.

hiroyuki-sato · 2017-06-09T00:23:53Z

@kares Thank you for your comment.

Yes, I tried it. The latest JRuby (9.1.10.0) got the same error.

Best regards.

Summaries

The Embulk use liquid gem.
Liquid 3.0.6 used File.dirname(__FILE__)
But 4.0.0 use __dir__.
Shopify/liquid@b31df0f#diff-1ac8d79c9833db7b503c16989031b253

Reproduce steps

Build embulk 0.8.20, update

Update liquid 3.0.6 -> 4.0.0
Update JRuby 9.1.5.0 -> 9.1.10.0

git clone https://github.com/embulk/embulk
cd embulk
git checkout -b v0.8.20 v0.8.20
perl -i -pe 's/9\.1\.5\.0/9.1.10.0/' build.gradle
perl -i -pe 's/3\.0\.6/4.0.0/' embulk-core/build.gradle
./gradlew cli

And test

./pkg/embulk-0.8.20.jar mkbundle dir_test_bundle
./pkg/embulk-0.8.20.jar example dir_test
./pkg/embulk-0.8.20.jar guess dir_test/seed.yml -o config.yml
./pkg/embulk-0.8.20.jar preview -b dir_test_bundle config.yml.liquid

Error.

undefined method `line_number' for #<String:0x652b3733>
    uri:classloader:/liquid/block_body.rb:99:in `block in render'
    org/jruby/RubyArray.java:1734:in `each'
    uri:classloader:/liquid/block_body.rb:75:in `render'
    uri:classloader:/liquid/template.rb:208:in `block in render'
    uri:classloader:/liquid/template.rb:242:in `with_profiling'
    uri:classloader:/liquid/template.rb:207:in `render'
    uri:classloader:/embulk/runner.rb:154:in `run_liquid'
    uri:classloader:/embulk/runner.rb:129:in `read_config'
    uri:classloader:/embulk/runner.rb:32:in `preview'
    uri:classloader:/embulk/command/embulk_run.rb:267:in `run'
    uri:classloader:/embulk/command/embulk_main.rb:2:in `<main>'
    org/jruby/RubyKernel.java:961:in `require'
    file:/private/tmp/embulk/pkg/embulk-0.8.20.jar!/embulk/command/embulk_bundle.rb:30:in `<main>'

Error: undefined method `line_number' for #<String:0x652b3733>

root cause

The 3.0.6 used File.dirname(__FILE__) for load tag classes.
But, 4.0.0 use __dir__. Then it points to another location if -b options use.

Liquid-3.0.6 lib/liquid.rb

Dir[File.dirname(__FILE__) + '/liquid/tags/*.rb'].each { |f| require f }

Liquid-4.0.0 lib/liquid.rb

Dir["#{__dir__}/liquid/tags/*.rb"].each { |f| require f }

Reference.

embulk/embulk#637

) ... if getAbsolutePath is used - which seems like its not necessary and we can pretty much just do what MRI does File.dirname(__FILE__)

__dir__ won't work with embed paths such as uri:classloader: (#4611)

kares · 2017-06-12T21:39:50Z

expected to be resolved with #4658

hiroyuki-sato · 2017-06-12T23:44:08Z

@kares Thank you for fixing this issue.

hiroyuki-sato mentioned this issue May 18, 2017

Update Liquid 4.0.0 again. embulk/embulk#637

Closed

3 tasks

stergiom mentioned this issue Jun 7, 2017

Inconsistency with __dir__ in jar #4649

Closed

kares added a commit to kares/jruby that referenced this issue Jun 9, 2017

__dir__ won't work with embed paths such as uri:classloader: (jruby#4611

404bed4

) ... if getAbsolutePath is used - which seems like its not necessary and we can pretty much just do what MRI does File.dirname(__FILE__)

kares mentioned this issue Jun 9, 2017

__dir__ won't work with embed paths such as uri:classloader: (#4611) #4658

Merged

kares added this to the JRuby 9.1.11.0 milestone Jun 11, 2017

kares added a commit to kares/jruby that referenced this issue Jun 11, 2017

[test] for __dir__ in (embed) jruby-complete.jar scenario (jruby#4611)

7a64aa6

kares added a commit to kares/jruby that referenced this issue Jun 11, 2017

[test] for __dir__ in (embed) jruby-complete.jar scenario (jruby#4611)

1460f59

kares added a commit that referenced this issue Jun 12, 2017

Merge pull request #4658 from kares/test-dir-in-jar

0190ec8

__dir__ won't work with embed paths such as uri:classloader: (#4611)

kares closed this as completed Jun 12, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

GitHub Sponsors

[Q] dir variable in jar file. #4611

[Q] dir variable in jar file. #4611

hiroyuki-sato commented May 17, 2017

enebo commented May 17, 2017

hiroyuki-sato commented May 18, 2017

kares commented Jun 8, 2017

kares commented Jun 8, 2017

headius commented Jun 8, 2017

hiroyuki-sato commented Jun 9, 2017

kares commented Jun 12, 2017

hiroyuki-sato commented Jun 12, 2017

[Q] __dir__ variable in jar file. #4611

[Q] __dir__ variable in jar file. #4611

Comments

hiroyuki-sato commented May 17, 2017

1. environment

2. expected and actual behavior

2.1 Expected Behavior

2.2 Actual Behavior

3. Question

4. Details

5. Test codes.

enebo commented May 17, 2017

hiroyuki-sato commented May 18, 2017

kares commented Jun 8, 2017

kares commented Jun 8, 2017

headius commented Jun 8, 2017

hiroyuki-sato commented Jun 9, 2017

Summaries

Reproduce steps

root cause

Reference.

kares commented Jun 12, 2017

hiroyuki-sato commented Jun 12, 2017

[Q] dir variable in jar file. #4611

[Q] dir variable in jar file. #4611