Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

jruby 9.2.0.0 on s390x #5248

Closed
p-mongo opened this issue Jul 12, 2018 · 11 comments
Closed

jruby 9.2.0.0 on s390x #5248

p-mongo opened this issue Jul 12, 2018 · 11 comments
Milestone
Invalid or Duplicate

Comments

@p-mongo
Copy link

p-mongo commented Jul 12, 2018

I am trying to install jruby 9.2.0.0 on IBM s390x. uname -a is as follows:

Linux rhel72-z-dev.stuff 3.10.0-693.17.1.el7.s390x #1 SMP Sun Jan 14 10:38:29 EST 2018 s390x s390x s390x GNU/Linux

The first hurdle I ran into was that uname -m returns s390x and jruby then attempts to load ./lib/jni/s390x-Linux/libjffi-1.2.so which does not exist. Instead, as far as I can tell, the correct binary for this machine is ./lib/jni/ppc64le-Linux/libjffi-1.2.so. I modified the tarball by hand basically copying ppc64le-Linux to s390x-Linux but I am wondering if there is a way to tell jruby to reference ppc64le-Linux without tarball modifications.

Then, I unpacked the tarball and ran irb which produced the following:

$ irb
OpenJDK 64-Bit Zero VM warning: You have loaded library /home/u/jruby-9.2.0.0/lib/jni/s390x-Linux/libjffi-1.2.so which might have disabled stack guard. The VM will try to fix the stack guard now.
It's highly recommended that you fix the library with 'execstack -c ', or link it with '-z noexecstack'.
NotImplementedError: fstat unimplemented unsupported or native support failed to load; see http://wiki.jruby.org/Native-Libraries
  initialize at org/jruby/RubyIO.java:1036
        open at org/jruby/RubyIO.java:1177
  initialize at /home/u/jruby-9.2.0.0/lib/ruby/stdlib/irb/input-method.rb:141
  initialize at /home/u/jruby-9.2.0.0/lib/ruby/stdlib/irb/context.rb:70
  initialize at /home/u/jruby-9.2.0.0/lib/ruby/stdlib/irb.rb:410
       start at /home/u/jruby-9.2.0.0/lib/ruby/stdlib/irb.rb:381
       at /home/u/jruby-9.2.0.0/bin/irb:13

The message says libjffi was loaded, therefore I am not clear whether the troubleshooting instructions in https://github.com/jruby/jruby/wiki/Native-Libraries apply.

Regardless, they recommend the following:

The first line of attack is to pass -Xnative.verbose=true to JRuby itself so it will log errors that occur during startup of our FFI subsystem.

They don't say how to pass that option to jruby, thus I tried irb -Xnative.verbose=true and that didn't work either:

$ irb -Xnative.verbose=true
OpenJDK 64-Bit Zero VM warning: You have loaded library /home/u/jruby-9.2.0.0/lib/jni/s390x-Linux/libjffi-1.2.so which might have disabled stack guard. The VM will try to fix the stack guard now.
It's highly recommended that you fix the library with 'execstack -c ', or link it with '-z noexecstack'.
IRB::UnrecognizedSwitch: Unrecognized switch: -Xnative.verbose=true

At this point I don't know how to troubleshoot further.
@headius
Copy link
Member

headius commented Jul 12, 2018

Ahh ok this isn't too bad. I think the simplest way would be like we do for Solaris/SunOS or Darwin/MacOS and have a translation table that knows "s390x" is a "ppc64le" environment.

You can pass that option to JRuby via JRUBY_OPTS env, or if running a command you can do something like jruby -Xnative.verbose -S irb.

@headius
Copy link
Member

headius commented Jul 12, 2018

This is probably a jffi or jnr-ffi bug.

@headius headius added this to the JRuby 9.2.1.0 milestone Jul 12, 2018
@p-mongo
Copy link
Author

p-mongo commented Jul 12, 2018

I think this was my mistake as I was on a big endian system.

On the big endian system I received this message with the native.verbose set:

java.lang.UnsatisfiedLinkError: /home/u/jruby-9.2.0.0/lib/jni/s390x-Linux/libjffi-1.2.so: /home/u/jruby-9.2.0.0/lib/jni/s390x-Linux/libjffi-1.2.so: ELF file data encoding not big-endian (Possible cause: endianness mismatch)

... which is indeed quite helpful.

However switching to ppc64-Linux produced:

java.lang.UnsatisfiedLinkError: /home/u/jruby-9.2.0.0/lib/jni/s390x-Linux/libjffi-1.2.so: /home/u/jruby-9.2.0.0/lib/jni/s390x-Linux/libjffi-1.2.so: cannot open shared object file: No such file or directory (Possible cause: can't load Power PC 64-bit .so on a IBM System/390-bit platform)

As I don't know anything about either PPC64 or s390 I have no idea if this is meant to work at all.

On the little endian system uname -m returns ppc64le which is great, I am investigating that one further.

@p-mongo
Copy link
Author

p-mongo commented Jul 12, 2018

The little endian system comes with java 7 which makes 9.2.0.0 a no go.

Installing 9.1.17.0 produces the following:

$ rbenv install -v jruby-9.1.17.0
/tmp/ruby-build.20180712232339.56898 ~/pubfiles
Downloading jruby-dist-9.1.17.0-bin.tar.gz...
HTTP/1.1 200 OK
Content-Type: binary/octet-stream
Content-Length: 21422828
Connection: keep-alive
Date: Thu, 12 Jul 2018 13:42:04 GMT
Last-Modified: Tue, 24 Apr 2018 00:11:27 GMT
ETag: "f28d1fcbb60e550f3abd839102ad0069"
Accept-Ranges: bytes
Server: AmazonS3
Age: 34904
X-Cache: Hit from cloudfront
Via: 1.1 5a57d853ab1d61f9038a75f2a12c9421.cloudfront.net (CloudFront)
X-Amz-Cf-Id: iIpYCXlcfGdjnSEs_CP7QbVz8awF8xiI-bru9DMyYtk5uFyP_A5Ktw==

-> https://dqw8nmjcqpjn7.cloudfront.net/6a22f7bf8fef1a52530a9c9781a9d374ad07bbbef0d3d8e2af0ff5cbead0dfd5
jruby-9.1.17.0
jruby-9.1.17.0.tar.gz
/tmp/ruby-build.20180712232339.56898/jruby-9.1.17.0 /tmp/ruby-build.20180712232339.56898 ~/pubfiles
Installing jruby-9.1.17.0...
ERROR:  Could not find a valid gem 'jruby-launcher' (>= 0), here is why:
          Unable to download data from https://rubygems.org/ - Received fatal alert: protocol_version (https://api.rubygems.org/specs.4.8.gz)

BUILD FAILED (RedHatEnterpriseServer 7.1 using ruby-build 20180618-5-gad8f3a4)

Inspect or clean up the working tree at /tmp/ruby-build.20180712232339.56898
Results logged to /tmp/ruby-build.20180712232339.56898.log

Last 10 log lines:
Age: 34904
X-Cache: Hit from cloudfront
Via: 1.1 5a57d853ab1d61f9038a75f2a12c9421.cloudfront.net (CloudFront)
X-Amz-Cf-Id: iIpYCXlcfGdjnSEs_CP7QbVz8awF8xiI-bru9DMyYtk5uFyP_A5Ktw==

jruby-9.1.17.0
jruby-9.1.17.0.tar.gz
/tmp/ruby-build.20180712232339.56898/jruby-9.1.17.0 /tmp/ruby-build.20180712232339.56898 ~/pubfiles
ERROR:  Could not find a valid gem 'jruby-launcher' (>= 0), here is why:
          Unable to download data from https://rubygems.org/ - Received fatal alert: protocol_version (https://api.rubygems.org/specs.4.8.gz)

$ openssl version  
OpenSSL 1.0.1e-fips 11 Feb 2013

I'm guessing the openssl is too old. I think I will give up on the 390s at this juncture.

@p-mongo p-mongo closed this as completed Jul 12, 2018
@headius
Copy link
Member

headius commented Jul 12, 2018

Hmm...well we don't use the system OpenSSL...we have a look-alike library build atop Bouncy Castle. But this does indeed look like some crypto/network issue.

We're happy to help you keep investigating but for obvious reasons it's difficult for us to fix these things alone :-(

@p-mongo
Copy link
Author

p-mongo commented Jul 12, 2018

Okay, my boss doesn't want me to spend too much time on s390x thus if you want me to run some commands or follow instructions like the jffi stuff above I can do that but figuring out where in the ruby/jruby/java/os stack this protocol error is coming from is not something I'll be able to do on my own.

@p-mongo
Copy link
Author

p-mongo commented Jul 12, 2018 •
edited

I downloaded the tarball referenced by rbenv, untarred it and was able to get an irb shell with 9.1.17.0. No warnings or errors but it takes ruby forever to launch:

$ time gem --version
2.6.14.1
gem --version  20.45s user 20.79s system 573% cpu 7.191 total

@p-mongo
Copy link
Author

p-mongo commented Jul 12, 2018 •
edited

And now I have this:

$ irb
irb(main):001:0> require 'open-uri'
=> true
irb(main):002:0> open('https://rubygems.org').read.length
OpenSSL::SSL::SSLError: Received fatal alert: protocol_version
	from org/jruby/ext/openssl/SSLSocket.java:228:in `connect_nonblock'
	from /home/u/jruby-9.1.17.0/lib/ruby/stdlib/net/http.rb:938:in `connect'
	from /home/u/jruby-9.1.17.0/lib/ruby/stdlib/net/http.rb:868:in `do_start'
	from /home/u/jruby-9.1.17.0/lib/ruby/stdlib/net/http.rb:857:in `start'
	from /home/u/jruby-9.1.17.0/lib/ruby/stdlib/open-uri.rb:319:in `open_http'
	from /home/u/jruby-9.1.17.0/lib/ruby/stdlib/open-uri.rb:737:in `buffer_open'
	from /home/u/jruby-9.1.17.0/lib/ruby/stdlib/open-uri.rb:212:in `block in open_loop'
	from org/jruby/RubyKernel.java:1138:in `catch'
	from /home/u/jruby-9.1.17.0/lib/ruby/stdlib/open-uri.rb:210:in `open_loop'
	from /home/u/jruby-9.1.17.0/lib/ruby/stdlib/open-uri.rb:151:in `open_uri'
	from /home/u/jruby-9.1.17.0/lib/ruby/stdlib/open-uri.rb:717:in `open'
	from /home/u/jruby-9.1.17.0/lib/ruby/stdlib/open-uri.rb:35:in `open'
	from (irb):2:in `'
	from org/jruby/RubyKernel.java:995:in `eval'
	from org/jruby/RubyKernel.java:1316:in `loop'
	from org/jruby/RubyKernel.java:1138:in `catch'
	from org/jruby/RubyKernel.java:1138:in `catch'
	from /home/u/jruby-9.1.17.0/bin/irb:13:in `'
irb(main):003:0> open('https://python.org').read.length
OpenSSL::SSL::SSLError: Received fatal alert: protocol_version
	from org/jruby/ext/openssl/SSLSocket.java:228:in `connect_nonblock'
	from /home/u/jruby-9.1.17.0/lib/ruby/stdlib/net/http.rb:938:in `connect'
	from /home/u/jruby-9.1.17.0/lib/ruby/stdlib/net/http.rb:868:in `do_start'
	from /home/u/jruby-9.1.17.0/lib/ruby/stdlib/net/http.rb:857:in `start'
	from /home/u/jruby-9.1.17.0/lib/ruby/stdlib/open-uri.rb:319:in `open_http'
	from /home/u/jruby-9.1.17.0/lib/ruby/stdlib/open-uri.rb:737:in `buffer_open'
	from /home/u/jruby-9.1.17.0/lib/ruby/stdlib/open-uri.rb:212:in `block in open_loop'
	from org/jruby/RubyKernel.java:1138:in `catch'
	from /home/u/jruby-9.1.17.0/lib/ruby/stdlib/open-uri.rb:210:in `open_loop'
	from /home/u/jruby-9.1.17.0/lib/ruby/stdlib/open-uri.rb:151:in `open_uri'
	from /home/u/jruby-9.1.17.0/lib/ruby/stdlib/open-uri.rb:717:in `open'
	from /home/u/jruby-9.1.17.0/lib/ruby/stdlib/open-uri.rb:35:in `open'
	from (irb):3:in `'
	from org/jruby/RubyKernel.java:995:in `eval'
	from org/jruby/RubyKernel.java:1316:in `loop'
	from org/jruby/RubyKernel.java:1138:in `catch'
	from org/jruby/RubyKernel.java:1138:in `catch'
	from /home/u/jruby-9.1.17.0/bin/irb:13:in `'
irb(main):004:0> open('https://google.com').read.length
=> 10798

$ java -showversion
java version "1.7.0"
Java(TM) SE Runtime Environment (build pxl6470_27sr3fp10-20150708_01(SR3 FP10))
IBM J9 VM (build 2.7, JRE 1.7.0 Linux ppc64le-64 Compressed References 20150630_255653 (JIT enabled, AOT enabled)
J9VM - R27_Java727_SR3_20150630_2236_B255653
JIT  - tr.r13.java_20150623_94888.01
GC   - R27_Java727_SR3_20150630_2236_B255653_CMPRSS
J9CL - 20150630_255653)
JCL - 20150628_01 based on Oracle jdk7u85-b15

@p-mongo
Copy link
Author

p-mongo commented Jul 13, 2018 •
edited

Installed ibm-java-jre-8.0-5.17-ppc64le-archive.bin from https://developer.ibm.com/javasdk/downloads/sdk8/, it seems to work:

$ java -showversion 2>&1 |head -10
java version "1.8.0_171"
Java(TM) SE Runtime Environment (build 8.0.5.17 - pxl6480sr5fp17-20180627_01(SR5 FP17))
IBM J9 VM (build 2.9, JRE 1.8.0 Linux ppc64le-64-Bit Compressed References 20180626_390413 (JIT enabled, AOT enabled)
OpenJ9   - 5cdc604
OMR      - a24bc01
IBM      - 21870d6)
JCL - 20180619_01 based on Oracle jdk8u171-b11

Usage: java [-options] class [args...]
           (to execute a class)
$ jruby -v                        
jruby 9.1.17.0 (2.3.3) 2018-04-20 d8b1ff9 IBM J9 VM 2.9 on 8.0.5.17 - pxl6480sr5fp17-20180627_01(SR5 FP17) +jit [linux-ppc64le]
$ jruby -S irb 
irb(main):001:0> require 'open-uri'
=> true
irb(main):002:0> open('https://rubygems.org').read.length
=> 11427

And I got it working with rbenv as well.

And I can install 9.2.0.0 since I am now using java 8.

Thanks for your help @headius .

@p-mongo
Copy link
Author

p-mongo commented Jul 13, 2018 •
edited

For the big endian machine (s390x uname) I have no clue what to do though.

@p-mongo
Copy link
Author

p-mongo commented Jul 13, 2018 •
edited

My current impression is that for s390x (big endian native) I will need to build jffi first.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
Invalid or Duplicate
Development

No branches or pull requests
2 participants
@headius @p-mongo