Since JRuby-9.0.0.0 changes to classloaders are failing ruby-processing #3152

monkstone · 2015-07-19T14:07:39Z

JRuby is losing the magic that let ruby-processing just work:-
To make ruby-processing work it is important that ruby and java classes can be added somewhat dynamically (including those with native binaries), the recent changes to the class loaders in JRuby-9.0.0.0 are making it all but impossible to combine say video libraries (incorporating gstreamer and the like) and graphics (OPENGL). In some ways this is a continuation the issue that I reported earlier #3036 which can be fixed for simple cases by loading the jars onto the java classpath. Now if there were a new way of doing this, but incompatible with the old way I would be very interested as I am trying to create a new version of ruby-processing JRubyArt. Here is what I currently have to do add video library to JRubyArt (this is hard coded in the app and OK for core libraries.

    def video
      Dir["#{Processing::RP_CONFIG['sketchbook_path']}/libraries/video/library/\*.jar"]
    end

previously with ruby-processing we were able to add the jars from ruby and in our library _loader used the following hack to load the native binaries (even when not in a jar)

    # HACK: For pure java libraries, such as the ones that are available
    # on this page: http://processing.org/reference/libraries/index.html
    # that include native code, we mess with the 'Java ClassLoader', so that
    # you don't have to futz with your PATH. But it's probably bad juju.
    def load_java_library(library_name)
      library_name = library_name.to_sym
      return true if @loaded_libraries.include?(library_name)
      jpath = get_library_directory_path(library_name, 'jar')
      jars = get_library_paths(library_name, 'jar')
      return false if jars.empty?
      jars.each { |jar| require jar }
      platform_specific_library_paths = get_platform_specific_library_paths(jpath)
      platform_specific_library_paths = platform_specific_library_paths.select do |ppath|
        FileTest.directory?(ppath) && !Dir.glob(File.join(ppath, '*.{so,dll,jnilib}')).empty?
      end
      unless platform_specific_library_paths.empty?
        platform_specific_library_paths << java.lang.System.getProperty('java.library.path')
        new_library_path = platform_specific_library_paths.join(java.io.File.pathSeparator)
        java.lang.System.setProperty('java.library.path', new_library_path)
        field = java.lang.Class.for_name('java.lang.ClassLoader').get_declared_field('sys_paths')
        if field
          field.accessible = true
          field.set(java.lang.Class.for_name('java.lang.System').get_class_loader, nil)
        end
      end
      @loaded_libraries[library_name] = true
    end

mkristian · 2015-07-19T15:28:08Z

could you be more specific, please.

version 9.0.0.0.rc2 did revert the changes regarding the CLI interface of jruby putting its dynamic classloader into Thread.currentThread.contextClassLoader.

BTW jruby-9k is not 2.2 compliant

monkstone · 2015-07-19T15:36:30Z

I think my updating overlapped your response. With ruby-processing I still need load to natives to classpath (with latest SNAPSHOT), which I found conflicts with previous loading (jruby-1.7.**). So I did a hack to only load binaries that way for jruby-9.0.0.0, but this leaves me with incompatability. I'm not worried about 2.2 compliance nice to use some 2.1 stuff though.

mkristian · 2015-07-19T15:44:30Z

could you point me to the hack you are using for loading binaries with jruby-9k ?

monkstone · 2015-07-19T15:50:33Z

Here is is the hack see line 197 for my crude detection of jruby version (note we could arrive here from mri ruby and mostly do).

mkristian · 2015-07-19T16:19:56Z

thanx. is this the piece of code which works with jruby-1.7.* but not with jruby-9k https://github.com/jashkenas/ruby-processing/blob/master/lib/ruby-processing/library_loader.rb#L68
?

or is it https://github.com/jashkenas/ruby-processing/blob/master/lib/ruby-processing/library_loader.rb#L62 ?

any sample jar or so file which I can use for an isolated test ?

monkstone · 2015-07-19T19:00:31Z

Oh my god I thought I was going mad for bit, lots of things working that previously had not, including video capture, so thumbs up for reversal of classloader. In summary the major problem now appears to be the need to load the native OPENGL binaries into the java classpath for jruby-9.0.0.0 (this is needed for any sketch using processing P2D or P3D modes but not the default java_mode) so any test will need to emulate a P2D or P3D sketch. Simple P2D and P3D sketches also run OK. However when I try and include my arcball library for example (a jruby extension), then we get this failure:-

Java::JavaLang::RuntimeException
org.jruby.exceptions.RaiseException: (NameError) cannot load Java class processing.arcball.Arcball
    com.jogamp.common.util.awt.AWTEDTExecutor.invoke(AWTEDTExecutor.java:58)
    jogamp.opengl.awt.AWTThreadingPlugin.invokeOnOpenGLThread(AWTThreadingPlugin.java:103)
    jogamp.opengl.ThreadingImpl.invokeOnOpenGLThread(ThreadingImpl.java:206)
    javax.media.opengl.Threading.invokeOnOpenGLThread(Threading.java:172)
    javax.media.opengl.Threading.invoke(Threading.java:191)
    javax.media.opengl.awt.GLCanvas.display(GLCanvas.java:541)
    processing.opengl.PJOGL.requestDraw(PJOGL.java:688)
    processing.opengl.PGraphicsOpenGL.requestDraw(PGraphicsOpenGL.java:1651)
    processing.core.PApplet.run(PApplet.java:2256)
    java.lang.Thread.run(Thread.java:745)

Or say for example we try and run a video capture sketch with P2D (OPENGL rendering)
and we get this:-

Java::JavaLang::RuntimeException
org.jruby.exceptions.RaiseException: (NameError) Capture not found in packages processing.video; last error: cannot link Java class processing.video.Capture, probable missing dependency: org/gstreamer/elements/BufferDataAppSink$Listener
    com.jogamp.common.util.awt.AWTEDTExecutor.invoke(AWTEDTExecutor.java:58)
    jogamp.opengl.awt.AWTThreadingPlugin.invokeOnOpenGLThread(AWTThreadingPlugin.java:103)
    jogamp.opengl.ThreadingImpl.invokeOnOpenGLThread(ThreadingImpl.java:206)
    javax.media.opengl.Threading.invokeOnOpenGLThread(Threading.java:172)
    javax.media.opengl.Threading.invoke(Threading.java:191)
    javax.media.opengl.awt.GLCanvas.display(GLCanvas.java:541)
    processing.opengl.PJOGL.requestDraw(PJOGL.java:688)
    processing.opengl.PGraphicsOpenGL.requestDraw(PGraphicsOpenGL.java:1651)
    processing.core.PApplet.run(PApplet.java:2256)
    java.lang.Thread.run(Thread.java:745)

the same sketch works fine and dandy if we use Java2D rendering, that is OK for this sketch but won't allow us to do GLSL filtering for example. Problem is creating a suitable test, although I think possibly related errors have been reported for Swing apps?

monkstone · 2015-07-20T06:29:18Z

@mkristian OK here is the test https://github.com/monkstone/animationloop which illustrates the problem of loading my ArcBall extension. I have not done any fancy automation I'm afraid you just need to edit runner.rb to test the permutations (runner is required for adding jars to classpath). Also I realise now that my HACK added all the processing core jars to the classpath (my filter wasn't working because I was using tap) not just the binaries (but this was probably required anyway).

mkristian · 2015-07-20T15:51:40Z

@monkstone thanx for the repo - I am looking what I can find now

mkristian · 2015-07-20T17:10:29Z

@monkstone this is what I found so far to get your sketch.rb working: https://gist.github.com/mkristian/9ce9eece566c1ac03301

there is still some strange things going on and I think overall the code should work with jruby-1.7 as well jruby-9k as is.

please note the public init method on Rarcball which I needed to add to avoid such errors https://gist.github.com/mkristian/9ce9eece566c1ac03301#file-error - @enebo any idea why this inner class can not access this package private static init method ?

enebo · 2015-07-20T19:30:55Z

As an added note to @mkristian question. We realized this behavior for @JRubyMethod has always existed and we are only seeing it due to the other issues associated with loading. I do not think it is a bad idea for us to recommend public but we should probably fix this in both 1.7.x and 9k.x

monkstone · 2015-07-20T19:42:42Z

@mkristian that seems to solve that one now, it looks as though I will need to look at the other loading error, which might not be so simple to model (and fix). That is the loading of the video library jars that work fine when sketch is of the Java2D type but fails when the sketch is of OPENGL type.

mkristian · 2015-07-20T20:21:28Z

@monkstone the fact that you can just run 'sketch.rb' without the runner with jruby-1.7.x is still enough "example" to see what is going wrong with "requiring" jars in 9k. once this is fixed (hopefully soon) then we can look into the further problems of yours.

closing it on the teardown on the runtime did cause unexpected behaviour executing jruby via CLI fixes #3149 and #3152

mkristian · 2015-07-20T21:06:01Z

@monkstone the current master can run sketch.rb as is and I think this is a good starting point to look into your other sketches. thanx again for repo - it was very helpful

monkstone · 2015-07-21T18:55:53Z

Excellent @mkristian that seems to be problem solved as far as I can tell, the classpath issue had put me off creating a version of JRubyArt where sketches run without a runner (for jruby) but that is a possibility again, and in the future ruby-processing also could be updated to use jruby-9.0.0.0.

enebo added this to the JRuby 9.0.0.0 milestone Jul 20, 2015

enebo added the JRuby 9000 label Jul 20, 2015

mkristian added a commit that referenced this issue Jul 20, 2015

do close jrubyclassloader on ScriptingContainer#terminate only

e3f84a0

closing it on the teardown on the runtime did cause unexpected behaviour executing jruby via CLI fixes #3149 and #3152

mkristian closed this as completed Jul 20, 2015

monkstone mentioned this issue Aug 12, 2015

Sketches not starting with Processing b3 ruby-processing/JRubyArt#9

Closed

mkristian mentioned this issue Oct 8, 2015

Running threads stay running after teardown #3313

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

GitHub Sponsors

Since JRuby-9.0.0.0 changes to classloaders are failing ruby-processing #3152

Since JRuby-9.0.0.0 changes to classloaders are failing ruby-processing #3152

monkstone commented Jul 19, 2015

mkristian commented Jul 19, 2015

monkstone commented Jul 19, 2015

mkristian commented Jul 19, 2015

monkstone commented Jul 19, 2015

mkristian commented Jul 19, 2015

monkstone commented Jul 19, 2015

monkstone commented Jul 20, 2015

mkristian commented Jul 20, 2015

mkristian commented Jul 20, 2015

enebo commented Jul 20, 2015

monkstone commented Jul 20, 2015

mkristian commented Jul 20, 2015

mkristian commented Jul 20, 2015

monkstone commented Jul 21, 2015

Since JRuby-9.0.0.0 changes to classloaders are failing ruby-processing #3152

Since JRuby-9.0.0.0 changes to classloaders are failing ruby-processing #3152

Comments

monkstone commented Jul 19, 2015

mkristian commented Jul 19, 2015

monkstone commented Jul 19, 2015

mkristian commented Jul 19, 2015

monkstone commented Jul 19, 2015

mkristian commented Jul 19, 2015

monkstone commented Jul 19, 2015

monkstone commented Jul 20, 2015

mkristian commented Jul 20, 2015

mkristian commented Jul 20, 2015

enebo commented Jul 20, 2015

monkstone commented Jul 20, 2015

mkristian commented Jul 20, 2015

mkristian commented Jul 20, 2015

monkstone commented Jul 21, 2015