Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

File::Stat#dev_minor throws error on IBM AIX Java7_64; looks like it has to do with jffi #5097

Closed
Taywee opened this issue Mar 20, 2018 · 24 comments
Closed

File::Stat#dev_minor throws error on IBM AIX Java7_64; looks like it has to do with jffi #5097

Taywee opened this issue Mar 20, 2018 · 24 comments
Milestone
JRuby 9.2.7.0

Comments

@Taywee
Copy link

Taywee commented Mar 20, 2018

Environment

$ uname -a
AIX *** 1 7 00F614B34C00 powerpc AIX

$ oslevel
7.1.0.0
$ /usr/java7_64/bin/java -version
java version "1.7.0"
Java(TM) SE Runtime Environment (build pap6470sr10fp10-20170726_05(SR10 FP10))
IBM J9 VM (build 2.6, JRE 1.7.0 AIX ppc64-64 Compressed References 20170718_357001 (JIT enabled, AOT enabled)
J9VM - R26_Java726_SR10_20170718_1208_B357001
JIT  - r11_20170718_357001
GC   - R26_Java726_SR10_20170718_1208_B357001_CMPRSS
J9CL - 20170718_357001)
JCL - 20170725_01 based on Oracle jdk7u151-b14
$ /usr/java7_64/bin/java -Djruby.native.verbose=true -jar ./jruby-complete-9.1.16.0.jar -v
jruby 9.1.16.0 (2.3.3) 2018-02-21 8f3f95a IBM J9 VM 2.6 on pap6470sr10fp10-20170726_05 (SR10 FP10) +jit [aix-ppc64]

Expected Behavior

File::Stat#dev_minor should not raise an exception

Actual Behavior

$ /usr/java7_64/bin/java -Djruby.native.verbose=true -jar ./jruby-complete-9.1.16.0.jar -e 'p File.stat("/etc/passwd").dev_minor'
Failed to load native POSIX impl; falling back on Java impl. Stacktrace follows.
java.lang.UnsatisfiedLinkError: could not load FFI provider jnr.ffi.provider.jffi.Provider
        at jnr.ffi.provider.InvalidRuntime.newLoadError(InvalidRuntime.java:101)
        at jnr.ffi.provider.InvalidRuntime.addressSize(InvalidRuntime.java:82)
        at jnr.posix.POSIXFactory.libraries(POSIXFactory.java:183)
        at jnr.posix.POSIXFactory.access$200(POSIXFactory.java:16)
        at jnr.posix.POSIXFactory$DefaultLibCProvider$SingletonHolder.<clinit>(POSIXFactory.java:279)
        at java.lang.J9VMInternals.initializeImpl(Native Method)
        at java.lang.J9VMInternals.initialize(J9VMInternals.java:235)
        at jnr.posix.POSIXFactory$DefaultLibCProvider.getLibC(POSIXFactory.java:283)
        at jnr.posix.BaseNativePOSIX.<init>(BaseNativePOSIX.java:37)
        at jnr.posix.AixPOSIX.<init>(AixPOSIX.java:44)
        at jnr.posix.POSIXFactory.loadAixPOSIX(POSIXFactory.java:167)
        at jnr.posix.POSIXFactory.loadNativePOSIX(POSIXFactory.java:133)
        at jnr.posix.POSIXFactory.loadPOSIX(POSIXFactory.java:90)
        at jnr.posix.LazyPOSIX.loadPOSIX(LazyPOSIX.java:38)
        at jnr.posix.LazyPOSIX.posix(LazyPOSIX.java:32)
        at jnr.posix.LazyPOSIX.isNative(LazyPOSIX.java:398)
        at org.jruby.RubyGlobal.initSTDIO(RubyGlobal.java:279)
        at org.jruby.RubyGlobal.createGlobals(RubyGlobal.java:203)
        at org.jruby.Ruby.init(Ruby.java:1206)
        at org.jruby.Ruby.newInstance(Ruby.java:341)
        at org.jruby.Main.internalRun(Main.java:271)
        at org.jruby.Main.run(Main.java:232)
        at org.jruby.Main.main(Main.java:204)
Caused by: java.lang.ExceptionInInitializerError
        at java.lang.J9VMInternals.initialize(J9VMInternals.java:257)
        at jnr.ffi.provider.jffi.NativeRuntime.getInstance(NativeRuntime.java:58)
        at jnr.ffi.provider.jffi.Provider.<init>(Provider.java:29)
        at java.lang.J9VMInternals.newInstanceImpl(Native Method)
        at java.lang.Class.newInstance(Class.java:1781)
        at jnr.ffi.provider.FFIProvider$SystemProviderSingletonHolder.getInstance(FFIProvider.java:68)
        at jnr.ffi.provider.FFIProvider$SystemProviderSingletonHolder.<clinit>(FFIProvider.java:57)
        at java.lang.J9VMInternals.initializeImpl(Native Method)
        at java.lang.J9VMInternals.initialize(J9VMInternals.java:235)
        at jnr.ffi.provider.FFIProvider.getSystemProvider(FFIProvider.java:35)
        at jnr.ffi.Runtime$SingletonHolder.<clinit>(Runtime.java:82)
        at java.lang.J9VMInternals.initializeImpl(Native Method)
        at java.lang.J9VMInternals.initialize(J9VMInternals.java:235)
        at jnr.ffi.Runtime.getSystemRuntime(Runtime.java:67)
        ... 21 more
Caused by: java.lang.IllegalStateException: Cause already initialized
        at java.lang.Throwable.setCause(Throwable.java:361)
        at java.lang.Throwable.initCause(Throwable.java:339)
        at com.kenai.jffi.Type$Builtin.lookupTypeInfo(Type.java:252)
        at com.kenai.jffi.Type$Builtin.getTypeInfo(Type.java:237)
        at com.kenai.jffi.Type.resolveSize(Type.java:155)
        at com.kenai.jffi.Type.size(Type.java:138)
        at jnr.ffi.provider.jffi.NativeRuntime$TypeDelegate.size(NativeRuntime.java:187)
        at jnr.ffi.provider.AbstractRuntime.<init>(AbstractRuntime.java:48)
        at jnr.ffi.provider.jffi.NativeRuntime.<init>(NativeRuntime.java:66)
        at jnr.ffi.provider.jffi.NativeRuntime.<init>(NativeRuntime.java:41)
        at jnr.ffi.provider.jffi.NativeRuntime$SingletonHolder.<clinit>(NativeRuntime.java:62)
        at java.lang.J9VMInternals.initializeImpl(Native Method)
        at java.lang.J9VMInternals.initialize(J9VMInternals.java:235)
        ... 34 more
NotImplementedError: stat.st_dev unsupported or native support failed to load; see http://wiki.jruby.org/Native-Libraries
  dev_minor at org/jruby/RubyFileStat.java:250
     <main> at -e:1

I don't presently have a 32-bit Java to test with. This looks like it could be related to some of the issues reported in #3495 and #2729
@Taywee
Copy link
Author

Taywee commented Mar 21, 2018 •
edited
Loading

Note that if this is an issue with jffi that I can help with, I'd be glad to. I can have an AIX 64-bit build server built out that would be useable for compilation, running tests, and making changes. If XL C/C++ is required, it might be a bit more work, as I'm not sure if we have a license for it; if GCC is good enough, it shouldn't be too much effort at all.

@headius
Copy link
Member

headius commented Mar 26, 2018

@Taywee We will definitely need assistance 😄

The actual error is being obscured by this IllegalStateException. That seems to be caused by us attempting to initialize the cause of an exception at Type$Builtin.lookupTypeInfo(Type.java:252) which has already been initialized. The first step would be removing that call (or better, raising a new error with the existing one as its cause) so we can see the actual problem.

You can stop by #jruby or #jnr on Freenode IRC or the jruby/jruby Gitter channel if you need help building and testing jffi.

@headius
Copy link
Member

headius commented Mar 26, 2018

@Taywee C/C++ compiler will be necessary for running jffi's test, but should not be necessary for a build (we keep a binary for AIX ppc64 in the repo). The process for plugging a new jffi into JRuby goes like this:

  1. From the jffi clone, mvn install to install a snapshot of your modified jffi into the local Maven repo
  2. Modify JRuby's root pom.rb so the jffi snapshot version is specified, e.g. `'jffi.version' => '1.2.17-SNAPSHOT'
  3. From the JRuby dir, build JRuby with mvn clean package

Once you've modified the pom.rb file, only steps 2 and 3 are necessary to rebuild JRuby with an updated jffi snapshot.

@Taywee
Copy link
Author

Taywee commented Apr 24, 2018

@headius Sorry about going so long without a reply. Got pulled into other tasks for a while. Looks like that other error is caused by a typo in jffi, in which it catches an error and looks like it intends to throw an error with it set as the cause in line 250 of src/main/java/com/kenai/jffi/Type.java:

            } catch (Throwable error) {
                UnsatisfiedLinkError ule = new UnsatisfiedLinkError("could not get native definition for type: " + nativeType);
                error.initCause(error);
                throw ule;
            }

Where it should probably be:

            } catch (Throwable error) {
                UnsatisfiedLinkError ule = new UnsatisfiedLinkError("could not get native definition for type: " + nativeType);
                ule.initCause(error);
                throw ule;
            }

I've corrected that in my local branch and am going to rebuild it and see what I can do about testing on AIX.

@Taywee
Copy link
Author

Taywee commented Apr 24, 2018

Success getting it built with these changes:

$ /usr/java7_64/bin/java -Djruby.native.verbose=true -jar ./jruby.jar -v
jruby 9.1.17.0-SNAPSHOT (2.3.3) 2018-04-24 d8b1ff9 IBM J9 VM 2.6 on pap6470sr10fp10-20170726_05 (SR10 FP10) +jit [aix-ppc64]

And now the new error:

$ /usr/java7_64/bin/java -Djruby.native.verbose=true -jar ./jruby.jar -e 'p File.stat("/etc/passwd").dev_minor'
Failed to load native POSIX impl; falling back on Java impl. Stacktrace follows.
java.lang.UnsatisfiedLinkError: could not load FFI provider jnr.ffi.provider.jffi.Provider
        at jnr.ffi.provider.InvalidRuntime.newLoadError(InvalidRuntime.java:101)
        at jnr.ffi.provider.InvalidRuntime.addressSize(InvalidRuntime.java:82)
        at jnr.posix.POSIXFactory.libraries(POSIXFactory.java:183)
        at jnr.posix.POSIXFactory.access$200(POSIXFactory.java:16)
        at jnr.posix.POSIXFactory$DefaultLibCProvider$SingletonHolder.<clinit>(POSIXFactory.java:279)
        at java.lang.J9VMInternals.initializeImpl(Native Method)
        at java.lang.J9VMInternals.initialize(J9VMInternals.java:235)
        at jnr.posix.POSIXFactory$DefaultLibCProvider.getLibC(POSIXFactory.java:283)
        at jnr.posix.BaseNativePOSIX.<init>(BaseNativePOSIX.java:37)
        at jnr.posix.AixPOSIX.<init>(AixPOSIX.java:44)
        at jnr.posix.POSIXFactory.loadAixPOSIX(POSIXFactory.java:167)
        at jnr.posix.POSIXFactory.loadNativePOSIX(POSIXFactory.java:133)
        at jnr.posix.POSIXFactory.loadPOSIX(POSIXFactory.java:90)
        at jnr.posix.LazyPOSIX.loadPOSIX(LazyPOSIX.java:38)
        at jnr.posix.LazyPOSIX.posix(LazyPOSIX.java:32)
        at jnr.posix.LazyPOSIX.isNative(LazyPOSIX.java:398)
        at org.jruby.RubyGlobal.initSTDIO(RubyGlobal.java:279)
        at org.jruby.RubyGlobal.createGlobals(RubyGlobal.java:203)
        at org.jruby.Ruby.init(Ruby.java:1206)
        at org.jruby.Ruby.newInstance(Ruby.java:341)
        at org.jruby.Main.internalRun(Main.java:271)
        at org.jruby.Main.run(Main.java:232)
        at org.jruby.Main.main(Main.java:204)
Caused by: java.lang.UnsatisfiedLinkError: could not get native definition for type: POINTER
        at com.kenai.jffi.Type$Builtin.lookupTypeInfo(Type.java:251)
        at com.kenai.jffi.Type$Builtin.getTypeInfo(Type.java:237)
        at com.kenai.jffi.Type.resolveSize(Type.java:155)
        at com.kenai.jffi.Type.size(Type.java:138)
        at jnr.ffi.provider.jffi.NativeRuntime$TypeDelegate.size(NativeRuntime.java:187)
        at jnr.ffi.provider.AbstractRuntime.<init>(AbstractRuntime.java:48)
        at jnr.ffi.provider.jffi.NativeRuntime.<init>(NativeRuntime.java:66)
        at jnr.ffi.provider.jffi.NativeRuntime.<init>(NativeRuntime.java:41)
        at jnr.ffi.provider.jffi.NativeRuntime$SingletonHolder.<clinit>(NativeRuntime.java:62)
        at java.lang.J9VMInternals.initializeImpl(Native Method)
        at java.lang.J9VMInternals.initialize(J9VMInternals.java:235)
        at jnr.ffi.provider.jffi.NativeRuntime.getInstance(NativeRuntime.java:58)
        at jnr.ffi.provider.jffi.Provider.<init>(Provider.java:29)
        at java.lang.J9VMInternals.newInstanceImpl(Native Method)
        at java.lang.Class.newInstance(Class.java:1781)
        at jnr.ffi.provider.FFIProvider$SystemProviderSingletonHolder.getInstance(FFIProvider.java:68)
        at jnr.ffi.provider.FFIProvider$SystemProviderSingletonHolder.<clinit>(FFIProvider.java:57)
        at java.lang.J9VMInternals.initializeImpl(Native Method)
        at java.lang.J9VMInternals.initialize(J9VMInternals.java:235)
        at jnr.ffi.provider.FFIProvider.getSystemProvider(FFIProvider.java:35)
        at jnr.ffi.Runtime$SingletonHolder.<clinit>(Runtime.java:82)
        at java.lang.J9VMInternals.initializeImpl(Native Method)
        at java.lang.J9VMInternals.initialize(J9VMInternals.java:235)
        at jnr.ffi.Runtime.getSystemRuntime(Runtime.java:67)
        ... 21 more
Caused by: java.lang.UnsatisfiedLinkError: java.lang.UnsatisfiedLinkError: could not locate stub library in jar file.  Tried [jni/ppc64-AIX/libjffi-1.2.a, /jni/ppc64-AIX/libjffi-1.2.a]
        at com.kenai.jffi.internal.StubLoader.getStubLibraryStream(StubLoader.java:407)
        at com.kenai.jffi.internal.StubLoader.loadFromJar(StubLoader.java:355)
        at com.kenai.jffi.internal.StubLoader.load(StubLoader.java:258)
        at com.kenai.jffi.internal.StubLoader.<clinit>(StubLoader.java:444)
        at java.lang.J9VMInternals.initializeImpl(Native Method)
        at java.lang.J9VMInternals.initialize(J9VMInternals.java:235)
        at java.lang.Class.forNameImpl(Native Method)
        at java.lang.Class.forName(Class.java:302)
        at com.kenai.jffi.Init.load(Init.java:68)
        at com.kenai.jffi.Foreign$InstanceHolder.getInstanceHolder(Foreign.java:49)
        at com.kenai.jffi.Foreign$InstanceHolder.<clinit>(Foreign.java:45)
        at java.lang.J9VMInternals.initializeImpl(Native Method)
        at java.lang.J9VMInternals.initialize(J9VMInternals.java:235)
        at com.kenai.jffi.Foreign.getInstance(Foreign.java:103)
        at com.kenai.jffi.Type$Builtin.lookupTypeInfo(Type.java:242)
        at com.kenai.jffi.Type$Builtin.getTypeInfo(Type.java:237)
        at com.kenai.jffi.Type.resolveSize(Type.java:155)
        at com.kenai.jffi.Type.size(Type.java:138)
        at jnr.ffi.provider.jffi.NativeRuntime$TypeDelegate.size(NativeRuntime.java:187)
        at jnr.ffi.provider.AbstractRuntime.<init>(AbstractRuntime.java:48)
        at jnr.ffi.provider.jffi.NativeRuntime.<init>(NativeRuntime.java:66)
        at jnr.ffi.provider.jffi.NativeRuntime.<init>(NativeRuntime.java:41)
        at jnr.ffi.provider.jffi.NativeRuntime$SingletonHolder.<clinit>(NativeRuntime.java:62)
        at java.lang.J9VMInternals.initializeImpl(Native Method)
        at java.lang.J9VMInternals.initialize(J9VMInternals.java:235)
        at jnr.ffi.provider.jffi.NativeRuntime.getInstance(NativeRuntime.java:58)
        at jnr.ffi.provider.jffi.Provider.<init>(Provider.java:29)
        at java.lang.J9VMInternals.newInstanceImpl(Native Method)
        at java.lang.Class.newInstance(Class.java:1781)
        at jnr.ffi.provider.FFIProvider$SystemProviderSingletonHolder.getInstance(FFIProvider.java:68)
        at jnr.ffi.provider.FFIProvider$SystemProviderSingletonHolder.<clinit>(FFIProvider.java:57)
        at java.lang.J9VMInternals.initializeImpl(Native Method)
        at java.lang.J9VMInternals.initialize(J9VMInternals.java:235)
        at jnr.ffi.provider.FFIProvider.getSystemProvider(FFIProvider.java:35)
        at jnr.ffi.Runtime$SingletonHolder.<clinit>(Runtime.java:82)
        at java.lang.J9VMInternals.initializeImpl(Native Method)
        at java.lang.J9VMInternals.initialize(J9VMInternals.java:235)
        at jnr.ffi.Runtime.getSystemRuntime(Runtime.java:67)
        at jnr.posix.POSIXFactory.libraries(POSIXFactory.java:183)
        at jnr.posix.POSIXFactory.access$200(POSIXFactory.java:16)
        at jnr.posix.POSIXFactory$DefaultLibCProvider$SingletonHolder.<clinit>(POSIXFactory.java:279)
        at java.lang.J9VMInternals.initializeImpl(Native Method)
        at java.lang.J9VMInternals.initialize(J9VMInternals.java:235)
        at jnr.posix.POSIXFactory$DefaultLibCProvider.getLibC(POSIXFactory.java:283)
        at jnr.posix.BaseNativePOSIX.<init>(BaseNativePOSIX.java:37)
        at jnr.posix.AixPOSIX.<init>(AixPOSIX.java:44)
        at jnr.posix.POSIXFactory.loadAixPOSIX(POSIXFactory.java:167)
        at jnr.posix.POSIXFactory.loadNativePOSIX(POSIXFactory.java:133)
        at jnr.posix.POSIXFactory.loadPOSIX(POSIXFactory.java:90)
        at jnr.posix.LazyPOSIX.loadPOSIX(LazyPOSIX.java:38)
        at jnr.posix.LazyPOSIX.posix(LazyPOSIX.java:32)
        at jnr.posix.LazyPOSIX.isNative(LazyPOSIX.java:398)
        at org.jruby.RubyGlobal.initSTDIO(RubyGlobal.java:279)
        at org.jruby.RubyGlobal.createGlobals(RubyGlobal.java:203)
        at org.jruby.Ruby.init(Ruby.java:1206)
        at org.jruby.Ruby.newInstance(Ruby.java:341)
        at org.jruby.Main.internalRun(Main.java:271)
        at org.jruby.Main.run(Main.java:232)
        at org.jruby.Main.main(Main.java:204)

        at com.kenai.jffi.Foreign.newLoadError(Foreign.java:72)
        at com.kenai.jffi.Foreign.access$300(Foreign.java:42)
        at com.kenai.jffi.Foreign$InValidInstanceHolder.getForeign(Foreign.java:98)
        at com.kenai.jffi.Foreign.getInstance(Foreign.java:103)
        at com.kenai.jffi.Type$Builtin.lookupTypeInfo(Type.java:242)
        ... 44 more
Caused by: java.lang.UnsatisfiedLinkError: java.lang.UnsatisfiedLinkError: could not locate stub library in jar file.  Tried [jni/ppc64-AIX/libjffi-1.2.a, /jni/ppc64-AIX/libjffi-1.2.a]
        at com.kenai.jffi.internal.StubLoader.getStubLibraryStream(StubLoader.java:407)
        at com.kenai.jffi.internal.StubLoader.loadFromJar(StubLoader.java:355)
        at com.kenai.jffi.internal.StubLoader.load(StubLoader.java:258)
        at com.kenai.jffi.internal.StubLoader.<clinit>(StubLoader.java:444)
        at java.lang.J9VMInternals.initializeImpl(Native Method)
        at java.lang.J9VMInternals.initialize(J9VMInternals.java:235)
        at java.lang.Class.forNameImpl(Native Method)
        at java.lang.Class.forName(Class.java:302)
        at com.kenai.jffi.Init.load(Init.java:68)
        at com.kenai.jffi.Foreign$InstanceHolder.getInstanceHolder(Foreign.java:49)
        at com.kenai.jffi.Foreign$InstanceHolder.<clinit>(Foreign.java:45)
        at java.lang.J9VMInternals.initializeImpl(Native Method)
        at java.lang.J9VMInternals.initialize(J9VMInternals.java:235)
        at com.kenai.jffi.Foreign.getInstance(Foreign.java:103)
        at com.kenai.jffi.Type$Builtin.lookupTypeInfo(Type.java:242)
        at com.kenai.jffi.Type$Builtin.getTypeInfo(Type.java:237)
        at com.kenai.jffi.Type.resolveSize(Type.java:155)
        at com.kenai.jffi.Type.size(Type.java:138)
        at jnr.ffi.provider.jffi.NativeRuntime$TypeDelegate.size(NativeRuntime.java:187)
        at jnr.ffi.provider.AbstractRuntime.<init>(AbstractRuntime.java:48)
        at jnr.ffi.provider.jffi.NativeRuntime.<init>(NativeRuntime.java:66)
        at jnr.ffi.provider.jffi.NativeRuntime.<init>(NativeRuntime.java:41)
        at jnr.ffi.provider.jffi.NativeRuntime$SingletonHolder.<clinit>(NativeRuntime.java:62)
        at java.lang.J9VMInternals.initializeImpl(Native Method)
        at java.lang.J9VMInternals.initialize(J9VMInternals.java:235)
        at jnr.ffi.provider.jffi.NativeRuntime.getInstance(NativeRuntime.java:58)
        at jnr.ffi.provider.jffi.Provider.<init>(Provider.java:29)
        at java.lang.J9VMInternals.newInstanceImpl(Native Method)
        at java.lang.Class.newInstance(Class.java:1781)
        at jnr.ffi.provider.FFIProvider$SystemProviderSingletonHolder.getInstance(FFIProvider.java:68)
        at jnr.ffi.provider.FFIProvider$SystemProviderSingletonHolder.<clinit>(FFIProvider.java:57)
        at java.lang.J9VMInternals.initializeImpl(Native Method)
        at java.lang.J9VMInternals.initialize(J9VMInternals.java:235)
        at jnr.ffi.provider.FFIProvider.getSystemProvider(FFIProvider.java:35)
        at jnr.ffi.Runtime$SingletonHolder.<clinit>(Runtime.java:82)
        at java.lang.J9VMInternals.initializeImpl(Native Method)
        at java.lang.J9VMInternals.initialize(J9VMInternals.java:235)
        at jnr.ffi.Runtime.getSystemRuntime(Runtime.java:67)
        at jnr.posix.POSIXFactory.libraries(POSIXFactory.java:183)
        at jnr.posix.POSIXFactory.access$200(POSIXFactory.java:16)
        at jnr.posix.POSIXFactory$DefaultLibCProvider$SingletonHolder.<clinit>(POSIXFactory.java:279)
        at java.lang.J9VMInternals.initializeImpl(Native Method)
        at java.lang.J9VMInternals.initialize(J9VMInternals.java:235)
        at jnr.posix.POSIXFactory$DefaultLibCProvider.getLibC(POSIXFactory.java:283)
        at jnr.posix.BaseNativePOSIX.<init>(BaseNativePOSIX.java:37)
        at jnr.posix.AixPOSIX.<init>(AixPOSIX.java:44)
        at jnr.posix.POSIXFactory.loadAixPOSIX(POSIXFactory.java:167)
        at jnr.posix.POSIXFactory.loadNativePOSIX(POSIXFactory.java:133)
        at jnr.posix.POSIXFactory.loadPOSIX(POSIXFactory.java:90)
        at jnr.posix.LazyPOSIX.loadPOSIX(LazyPOSIX.java:38)
        at jnr.posix.LazyPOSIX.posix(LazyPOSIX.java:32)
        at jnr.posix.LazyPOSIX.isNative(LazyPOSIX.java:398)
        at org.jruby.RubyGlobal.initSTDIO(RubyGlobal.java:279)
        at org.jruby.RubyGlobal.createGlobals(RubyGlobal.java:203)
        at org.jruby.Ruby.init(Ruby.java:1206)
        at org.jruby.Ruby.newInstance(Ruby.java:341)
        at org.jruby.Main.internalRun(Main.java:271)
        at org.jruby.Main.run(Main.java:232)
        at org.jruby.Main.main(Main.java:204)

        at com.kenai.jffi.internal.StubLoader.load(StubLoader.java:270)
        at com.kenai.jffi.internal.StubLoader.<clinit>(StubLoader.java:444)
        at java.lang.J9VMInternals.initializeImpl(Native Method)
        at java.lang.J9VMInternals.initialize(J9VMInternals.java:235)
        at java.lang.Class.forNameImpl(Native Method)
        at java.lang.Class.forName(Class.java:302)
        at com.kenai.jffi.Init.load(Init.java:68)
        at com.kenai.jffi.Foreign$InstanceHolder.getInstanceHolder(Foreign.java:49)
        at com.kenai.jffi.Foreign$InstanceHolder.<clinit>(Foreign.java:45)
        at java.lang.J9VMInternals.initializeImpl(Native Method)
        at java.lang.J9VMInternals.initialize(J9VMInternals.java:235)
        ... 46 more
NotImplementedError: stat.st_dev unsupported or native support failed to load; see http://wiki.jruby.org/Native-Libraries
  dev_minor at org/jruby/RubyFileStat.java:250
     <main> at -e:1

Now it looks like UnsatisfiedLinkErrors the whole way down.

@Taywee
Copy link
Author

Taywee commented Apr 24, 2018

@headius Are you sure you keep a PPC64-AIX? I don't see one in here, just the 32-bit one.

@headius
Copy link
Member

headius commented Apr 25, 2018

@Taywee Oh, perhaps I was mistaken!

The POINTER definition error may mean there's a file missing with appropriate defs for these types. I don't have the source in front of me, but search around for where POINTER is getting defined for other platforms.

@Taywee
Copy link
Author

Taywee commented Apr 25, 2018

The next error down looks like "Caused by: java.lang.UnsatisfiedLinkError: java.lang.UnsatisfiedLinkError: could not locate stub library in jar file. Tried [jni/ppc64-AIX/libjffi-1.2.a, /jni/ppc64-AIX/libjffi-1.2.a]", and it doesn't look like either exists. I've been trying to get jffi to build on an AIX64 machine, but I'm running into some errors with defines. Looks like libffi hasn't been tested on AIX with GCC, only XLC, so that could be related.

@Taywee
Copy link
Author

Taywee commented Apr 26, 2018

I managed to get jffi build, but it took some hacking:

-build-platform-jar:
     [echo] platform=ppc64-AIX
      [zip] Building zip: /home/tcr/jffi/dist/jffi-ppc64-AIX.jar

I'm not fully versed in JNI, though. I'll have to do a bit of testing to see if this works the way it should tomorrow.

@Taywee
Copy link
Author

Taywee commented Apr 26, 2018

Getting errors still with this jar in the mix, so I supposethe hacks I used were insufficient. I'll have to do more tinkering in here to get AIX working correctly with this.

@Taywee
Copy link
Author

Taywee commented Apr 30, 2018

@headius
I've gotten jffi compiling, installing locally, and being pulled into jruby.jar. Trying to run it on AIX with native behavior gives me this much shorter error (I am doing all my building with GCC):

$ /usr/java7_64/bin/java -Djruby.native.verbose=true -jar ~/jruby.jar -e 'p File.stat("/etc/passwd").dev_minor'
Apr 30, 2018 10:54:17 PM jnr.ffi.provider.jffi.NativeRuntime buildNativeTypeAliases
SEVERE: failed to load type aliases: java.lang.ClassNotFoundException: jnr.ffi.provider.jffi.platform.ppc64.aix.TypeAliases
Failed to load native POSIX impl; falling back on Java impl. Stacktrace follows.
java.lang.ExceptionInInitializerError
        at java.lang.J9VMInternals.initialize(J9VMInternals.java:257)
        at jnr.posix.POSIXFactory$DefaultLibCProvider.getLibC(POSIXFactory.java:283)
        at jnr.posix.BaseNativePOSIX.<init>(BaseNativePOSIX.java:37)
        at jnr.posix.AixPOSIX.<init>(AixPOSIX.java:44)
        at jnr.posix.POSIXFactory.loadAixPOSIX(POSIXFactory.java:167)
        at jnr.posix.POSIXFactory.loadNativePOSIX(POSIXFactory.java:133)
        at jnr.posix.POSIXFactory.loadPOSIX(POSIXFactory.java:90)
        at jnr.posix.LazyPOSIX.loadPOSIX(LazyPOSIX.java:38)
        at jnr.posix.LazyPOSIX.posix(LazyPOSIX.java:32)
        at jnr.posix.LazyPOSIX.isNative(LazyPOSIX.java:398)
        at org.jruby.RubyGlobal.initSTDIO(RubyGlobal.java:279)
        at org.jruby.RubyGlobal.createGlobals(RubyGlobal.java:203)
        at org.jruby.Ruby.init(Ruby.java:1206)
        at org.jruby.Ruby.newInstance(Ruby.java:341)
        at org.jruby.Main.internalRun(Main.java:271)
        at org.jruby.Main.run(Main.java:232)
        at org.jruby.Main.main(Main.java:204)
Caused by: java.lang.RuntimeException: invalid type: pid_t
        at jnr.ffi.provider.BadType.getNativeType(BadType.java:43)
        at jnr.ffi.provider.jffi.InvokerUtil.getMethodParameterNativeType(InvokerUtil.java:213)
        at jnr.ffi.provider.jffi.InvokerUtil.getParameterType(InvokerUtil.java:151)
        at jnr.ffi.provider.jffi.InvokerUtil.getParameterTypes(InvokerUtil.java:180)
        at jnr.ffi.provider.jffi.AsmLibraryLoader.generateInterfaceImpl(AsmLibraryLoader.java:166)
        at jnr.ffi.provider.jffi.AsmLibraryLoader.loadLibrary(AsmLibraryLoader.java:89)
        at jnr.ffi.provider.jffi.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:44)
        at jnr.ffi.LibraryLoader.load(LibraryLoader.java:325)
        at jnr.ffi.Library.loadLibrary(Library.java:127)
        at jnr.posix.POSIXFactory$DefaultLibCProvider$SingletonHolder.<clinit>(POSIXFactory.java:279)
        at java.lang.J9VMInternals.initializeImpl(Native Method)
        at java.lang.J9VMInternals.initialize(J9VMInternals.java:235)
        ... 16 more
NotImplementedError: stat.st_dev unsupported or native support failed to load; see http://wiki.jruby.org/Native-Libraries
  dev_minor at org/jruby/RubyFileStat.java:250
     <main> at -e:1

@Taywee
Copy link
Author

Taywee commented Apr 30, 2018 •
edited
Loading

Ah, I think I found the source of that error:

% grep -FR pid_t . | grep ppc                                                                                         [jnr-ffi] (master|✔)
./src/main/java/jnr/ffi/provider/jffi/platform/ppc/aix/TypeAliases.java:        m.put(TypeAlias.pid_t, NativeType.SINT);
./src/main/java/jnr/ffi/provider/jffi/platform/ppc/darwin/TypeAliases.java:        m.put(TypeAlias.pid_t, NativeType.SINT);
./src/main/java/jnr/ffi/provider/jffi/platform/ppc/linux/TypeAliases.java:        m.put(TypeAlias.pid_t, NativeType.SINT);
./src/main/java/jnr/ffi/provider/jffi/platform/ppc64/linux/TypeAliases.java:        m.put(TypeAlias.pid_t, NativeType.SINT);
./src/main/java/jnr/ffi/provider/jffi/platform/ppc64le/linux/TypeAliases.java:        m.put(TypeAlias.pid_t, NativeType.SINT);

jnr-ffi doesn't have a ppc64/aix platform; it has only linux on ppc64, so jnr-ffi will need changes of its own. Tomorrow, I'll work on trying to get jnr-ffi fixed and rolled into jffi correctly. If I end up getting this fixed, I'll make a few pull requests (one for the jnr-ffi AIX64 patch and one for jffi's AIX64 stuff, as it also needed some low-impact modifications to compile on AIX [mostly a couple changes to preprocessor defines to include correct headers]).

@Taywee
Copy link
Author

Taywee commented May 1, 2018

I've got the TypeAliases file written out for jnr-ffi and got that compiled and pulling in, and now I'm getting this kind of error:

Caused by: java.lang.VerifyError: JVMVRFY012 stack shape inconsistent; class=jnr/posix/AixLibC$jnr$ffi$0, method=crypt(Ljava/lang/CharSequence;Ljava/lang/CharSequence;)Ljava/lang/CharSequence;, pc=81
        at java.lang.J9VMInternals.verifyImpl(Native Method)
        at java.lang.J9VMInternals.verify(J9VMInternals.java:94)
        at java.lang.J9VMInternals.prepare(J9VMInternals.java:514)
        at java.lang.Class.getDeclaredConstructor(Class.java:611)
        at jnr.ffi.provider.jffi.AsmLibraryLoader.generateInterfaceImpl(AsmLibraryLoader.java:236)
        ... 23 more

I'm not completely sure why. Might be that I'm using a Java 8 JDK while running on a Java 7 JVM.

@Taywee
Copy link
Author

Taywee commented May 3, 2018

@headius I've opened related issue in the JFFI tracker, in jnr/jffi#53, as that's a much more appropriate place for it.

@Taywee
Copy link
Author

Taywee commented May 7, 2018 •
edited
Loading

Pull request jnr/jffi#54 appears to fix this issue, when pulled into jnr-ffi and jruby as appropriate:

$ /usr/java7_64/bin/java -Djruby.native.verbose=true -jar ~/jruby.jar -e 'p File.stat("/etc/passwd").dev_minor'
Successfully loaded native POSIX impl.
4

@headius
Copy link
Member

headius commented May 14, 2018

@Taywee That's great! I've merged in your changes. However we do not have access to an AIX machine, and this will require a new native binary for AIX. Can you provide that in another pull request?

@headius headius added this to the JRuby 9.2.0.0 milestone May 14, 2018
@Taywee
Copy link
Author

Taywee commented May 14, 2018

Pretty sure it's in there, in archive/jffi-ppc64-AIX.jar. This file gets pulled in when I do a mvn package of jnr and of jruby from any machine. I ran into a few tangentially-related issues during run related to missing platform constants, for which I've opened an issue and corresponding pull request on jnr constants. That patch allows more full use, as without it files can't be read (they are opened with the wrong mode).

@headius
Copy link
Member

headius commented May 14, 2018

Oh you're right...I scrolled right by it.

@enebo We will need to release this and probably do a round of jnr-* releases for 9.2.

@enebo enebo modified the milestones: JRuby 9.2.0.0, JRuby 9.2.1.0 May 24, 2018
@Taywee
Copy link
Author

Taywee commented Jun 21, 2018

@headius Just so you're aware, AIX functionality is dependent on another couple issues (with open pull requests) in other repositories: jnr/jnr-posix#118 closed by jnr/jnr-posix#117 to fix flock and timeval on AIX (the commit message explains some decisions in there), and jnr/jnr-constants#27 closed by jnr/jnr-constants#28.

It doesn't completely AIX up to speed, but when I run some of the jruby test suites with these changes pulled in, everything runs and upwards of 99% of the tests pass, which is good enough for my uses and many other people's. Many of the tests that didn't pass were due to me not running with a unicode locale available; some other ones were due to legitimate unfixed bugs, and I think some of the others were either AIX not respecting POSIX behavior or Ruby expecting common Unix behavior instead of true POSIX behavior, but I didn't take a lot of notes while doing these runs.

@headius
Copy link
Member

headius commented Oct 11, 2018

We have had AIX support updates recently...perhaps this can be retested?

@Taywee
Copy link
Author

Taywee commented Oct 11, 2018

Yes, I had a jruby-complete snapshot build integrating some of those AIX changes in the jnr dependencies and it appeared to fix most of the major issues. I'm not sure how that sits with jnr/jnr-constants#51 though. Some changes lost me my AIX development servers temporarily, so I'm going to have to wait until I can rebuild those to get those enums set up again.

@headius
Copy link
Member

headius commented Oct 11, 2018

@Taywee Ok keep us posted on that. Very glad to hear that most things are resolved!

@headius
Copy link
Member

headius commented Oct 12, 2018

I have pinged https://www.skytap.com about getting OSS access to their AIX on Power cloud...if that comes through maybe we'll have a means to test and build more regularly.

@headius headius modified the milestones: JRuby 9.2.1.0, JRuby 9.2.2.0 Oct 26, 2018
@enebo enebo modified the milestones: JRuby 9.2.2.0, JRuby 9.2.3.0 Nov 7, 2018
@enebo enebo modified the milestones: JRuby 9.2.5.0, JRuby 9.2.6.0 Dec 6, 2018
@enebo enebo modified the milestones: JRuby 9.2.6.0, JRuby 9.2.7.0 Dec 19, 2018
@enebo enebo modified the milestones: JRuby 9.2.7.0, JRuby 9.2.8.0 Jan 9, 2019
@headius headius modified the milestones: JRuby 9.2.8.0, JRuby 9.2.7.0 Aug 6, 2019
@headius
Copy link
Member

headius commented Aug 6, 2019

I'm going to call this fixed as of 9.2.7.0, since we have no way to access or test on AIX and have done many releases with updated JNR support since it was filed.

@Taywee maybe you can retest with a more recent JRuby and let us know what remains to do here? If there's still problems, please open a new issue.

@headius headius closed this as completed Aug 6, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
JRuby 9.2.7.0
Development

No branches or pull requests
3 participants
@headius @enebo @Taywee