Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JRuby can Not load JFFI library under IBM Power 64 with PLinux platform #1389

Closed
xialixl opened this issue Jan 9, 2014 · 32 comments
Closed

Comments

@xialixl
Copy link

xialixl commented Jan 9, 2014

Environments:
IBM Power ppc 64 bit , POWER7 architecture,
Red Hat Enterprise Linux Server release 6.3 (Santiago)
JRuby ver 1.6.8, jar file: jruby-complete-1.6.8.jar
Apache Hbase 0.95.2

In case of running Command from Hbase :
.../hbase-0.95.2/bin/hbase shell

We got Errors like following:

INFO  [main] Configuration.deprecation: hadoop.native.lib is deprecated. Instead, use io.native.lib.available
java.lang.RuntimeException: java.lang.UnsatisfiedLinkError: Could not locate stub library in jar file.  Tried [jni/ppc64-Linux/libjffi-1.0.so, /jni/ppc64-Linux/libjffi-1.0.so]
        at com.kenai.jffi.Foreign$InValidInstanceHolder.getForeign(Foreign.java:90)

My Analysis:
It looks like JRuby was going to load Java JFFI lib: libjffi-1.0.so which is supposed to exist under location jni/ppc64-Linux/libjffi-1.0.so inside the Jar 'ruby-complete-1.6.8.jar'.
for 1.6.8 version currently Jruby only has following kinds of JFFI lib for choices :

jni/Darwin/
jni/Darwin/libjffi-1.0.jnilib
jni/i386-FreeBSD/
jni/i386-FreeBSD/libjffi-1.0.so
jni/i386-Linux/
jni/i386-Linux/libjffi-1.0.so
jni/i386-OpenBSD/
jni/i386-OpenBSD/libjffi-1.0.so
jni/i386-SunOS/
jni/i386-SunOS/libjffi-1.0.so
jni/i386-Windows/
jni/i386-Windows/jffi-1.0.dll
jni/ppc-AIX/
jni/ppc-AIX/libjffi-1.0.a
jni/ppc-Linux/
jni/ppc-Linux/libjffi-1.0.so
jni/sparc-SunOS/
jni/sparc-SunOS/libjffi-1.0.so
jni/sparcv9-SunOS/
jni/sparcv9-SunOS/libjffi-1.0.so
jni/x86_64-FreeBSD/
jni/x86_64-FreeBSD/libjffi-1.0.so
jni/x86_64-Linux/
jni/x86_64-Linux/libjffi-1.0.so
jni/x86_64-OpenBSD/
jni/x86_64-OpenBSD/libjffi-1.0.so
jni/x86_64-SunOS/
jni/x86_64-SunOS/libjffi-1.0.so
jni/x86_64-Windows/
jni/x86_64-Windows/jffi-1.0.dll

Fix:
Since '/jni/ppc64-Linux/libjffi-1.0.so' is missed inside jar , I downloaded the right version of JFFI sources (https://github.com/jnr/jffi) and built them by IBM XL C Compiler instead of default GCC at IBM ppc64 platform , Added the output libjffi-1.0.so into the right place of the jruby-complete-1.6.8.jar. Now it works well at my ppc64 env.

Advice:
To support IBM ppc64 with Linux env , Could Jruby add JFFI library for ppc64 version into released Jruby jar files.

Thanks

@BanzaiMan
Copy link
Member

We generally don't have access to these hardware, so we ask users to provide us with binary files such as these.

Thanks.

@xialixl
Copy link
Author

xialixl commented Jan 21, 2014

Ok, I got the lib file and jar file that could provide you with jruby, but how I could share with you and the project , Fork the project ?
sorry I am quite new for using github .

@faisalmoeen
Copy link

I am facing the same issue. Xialixl, can you please send the files to fmorakzai@gmail.com ?

@mkristian
Copy link
Member

@xialixl sorry for the late reply: fork https://github.com/jnr/jffi and send a pull request

@ayappanec
Copy link

A pull request has been sent to jffi repo with both ppc64-Linux and ppc64le-Linux binaries. Now the question here is whether the binaries 'ppc64 (ppc64le) libjffi-1.0.so' files will be added to the already published jruby-complete-1.6.8.jar file ?

@mkristian
Copy link
Member

we can not republish released jar (at least on maven central this is not
possible), but if you have a local maven repo you could add the missing
libjffi.so in the jruby-complete file

@ayappanec
Copy link

Ok. We have this problem of locally building jruby-complete-1.6.8.jar file in ppc64 (Big Endian) systems
A issue has been opened ( https://jira.codehaus.org/browse/JRUBY-7201 )

@ayappanec
Copy link

Can any maintainers look into this issue ( jnr/jffi#12 )

@headius
Copy link
Member

headius commented Jan 23, 2015

Wow, apologies for missing this one for so long. The PR has been merged and we'll try to get releases of the whole JNR chain out for JRuby 1.7.x and 9k.pre2.

@headius headius closed this as completed Jan 23, 2015
@headius headius added this to the 9.0.0.0.pre2 milestone Jan 23, 2015
@headius
Copy link
Member

headius commented Jan 23, 2015

Oops...don't close until we have updated JNR-* in a JRuby release.

@headius headius reopened this Jan 23, 2015
@ayappanec
Copy link

Can this be added to upcoming JRuby 1.7.x release also ?

@ayappanec
Copy link

Any update for the above comment?

@gustavotemple
Copy link
Contributor

@ayappanec, did you see this issue?

jnr/jffi#15

@headius
Copy link
Member

headius commented Feb 25, 2015

I don't see why we couldn't update this in 1.7.x as well. @enebo and I know we need to update everything for next releases, which will involve fixing some regressions in the jnr-* chain of projects. Marking for 1.7.20.

@headius headius modified the milestones: JRuby 1.7.20, 9.0.0.0.pre2 Feb 25, 2015
@headius
Copy link
Member

headius commented Feb 26, 2015

jnr/jffi#16 details the now-mitigated showstopper preventing us from releaseing jffi 1.2.8. Should be able to get the family released this week.

@ayappanec
Copy link

I came to know about the problem you guys facing for releasing jffi 1.2.8. I hope this one will get into Jruby 1.7.20 & 9000. Thank you.

@enebo
Copy link
Member

enebo commented Mar 3, 2015

@headius was able to release jffi and dependent artifacts yesterday so the plan is to update master and jruby-1_7 before our next releases.

@headius
Copy link
Member

headius commented Mar 3, 2015

1.7 and master have both been updated for latest jnr-* projects.

If it's possible for someone on PLinux/ppc64 to do a build of jruby-1_7 branch and confirm it works properly, we'd appreciate it :-)

@headius headius closed this as completed Mar 3, 2015
@ayappanec
Copy link

Building jruby-1_7 branch fails in PLinux/ppc64 & PLinux/ppc64le due to unknown compilation issue.
"Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile) on project jruby-core: Compilation failure"

But the main problem here is we are not able to build jruby-complete jar individually due to missing artfifacts.
"Failed to execute goal on project jruby-complete: Could not resolve dependencies for project org.jruby:jruby-complete:bundle:1.7.20-SNAPSHOT: The following artifacts could not be resolved: org.jruby:jruby-core:jar:noasm:1.7.20-SNAPSHOT, org.jruby:jruby-stdlib:jar:1.7.20-SNAPSHOT: Could not find artifact org.jruby:jruby-core:jar:noasm:1.7.20-SNAPSHOT in sonatype (https://oss.sonatype.org/content/repositories/snapshots/)"

It seems like the artifacts are not yet available in sonatype repository.

@mkristian
Copy link
Member

to build jruby-complete you need to run from the basedir of jruby
mvn -Pcomplete
or mvn clean install -Pcomplete
if this is not passing beyond jruby-core then nothing furtehr down the road
will work since all those missing artifacts will be build on the way

On Wed, Mar 4, 2015 at 8:25 AM, ayappanec notifications@github.com wrote:

Building jruby-1_7 branch fails in PLinux/ppc64 & PLinux/ppc64le due to
unknown compilation issue.
"Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:3.1:compile
(default-compile) on project jruby-core: Compilation failure"

But the main problem here is we are not able to build jruby-complete jar
individually due to missing artfifacts.
"Failed to execute goal on project jruby-complete: Could not resolve
dependencies for project org.jruby:jruby-complete:bundle:1.7.20-SNAPSHOT:
The following artifacts could not be resolved:
org.jruby:jruby-core:jar:noasm:1.7.20-SNAPSHOT,
org.jruby:jruby-stdlib:jar:1.7.20-SNAPSHOT: Could not find artifact
org.jruby:jruby-core:jar:noasm:1.7.20-SNAPSHOT in sonatype (
https://oss.sonatype.org/content/repositories/snapshots/)"

It seems like the artifacts are not yet available in sonatype repository.


Reply to this email directly or view it on GitHub
#1389 (comment).

@ayappanec
Copy link

I tried both the options but they failed to get beyond jruby-core. Finally i went into the maven/jruby-complete directory and tried to build jruby-complete jar by issuing "mvn clean install" and there i faced this missing artifacts issue..

@mkristian
Copy link
Member

@headius
Copy link
Member

headius commented Mar 4, 2015

@ayappanec Please open a separate issue for your problems building JRuby on PLinux and we'll work through them separately. I suspect it's some oddity in how our build interacts with the JVM on that platform.

@ayappanec
Copy link

@mkristian It requires jruby-core,jruby-stdlib and other such jars also.Can you please place those jars also in sonatype?
@headius Sure. I will open a separate issue for building JRuby on PLinux

@mkristian
Copy link
Member

mkristian commented Mar 4, 2015 via email

@ayappanec
Copy link

I tried with the command mentioned above. its trying to download the jruby-core jar from sonatype but failed since the jar uploaded there has some number associated with the filename "jruby-stdlib-1.7.20-20150304.132127-1.jar"

@headius
Copy link
Member

headius commented Mar 4, 2015

@ayappanec Thank you...please let us know when you've opened the other issue. For confirming this issue right now, just download the 1.7.20 build that @mkristian provided above.

@ayappanec
Copy link

@headius I could see the jffi binaries for ppc64 & ppc64le in the jruby-complete jar. Thank you

@ayappanec
Copy link

@headius I opened a new issue for Jruby build failure (#2648)

@ayappanec
Copy link

It seems like we need ppc64le jnr-ffi TypeAliases and ruby ffi ppc64le platform specific types.conf as well to get the latest jruby working ppc64le. I could see the changes already went into the master branch of jnr-ffi & jruby. I don't know when the jnr-ffi next release is going to happen. If it happens before jruby release, then we need to change the jnr-ffi version to 2.0.3 in jruby to make use of it. Otherwise we have wait till further releases of jruby.

@headius
Copy link
Member

headius commented Apr 6, 2015

@ayappanec I'm largely paralyzed from improving ppc64le support right now without your help, so if there are additional issues we need to fix, I'm hoping to see PRs from you :-)

@ayappanec
Copy link

@headius Sure.. Right now #2790 & #1897 seems to be the major hurdle. #2647 is yet another one but we have openjdk as an option here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

8 participants