review (cleanup) boot -> standard JRuby extension loading #5205

kares · 2018-06-01T13:55:55Z

... while looking into jruby loading performance and making JI native, managed to review the boot process.
there was some legacy and obvious things that we do not need to parse or can afford to lazy load instead.

the most interesting part (some cleanup noise through the commits) is an extension loading support :
JRuby::Util.load_ext(...) or the more official JRuby.load_ext (ideally after require 'jruby')

avoiding JRuby.runtime, and JI proxy loading the bootstrap Java class, JRuby is able to boot 5+% faster.

note, that we already delivered a ~ 5% improvement in 9.2.0 (over 9.1) this is measured compared to 9.2.0

also this is very useful internally since self-reflecting (JI loading org.jruby pieces) leads to a weird internal state (there were already some work-arounds on that from e.g. subclasses returning weird junk due UNDEF).

NOTE: things to consider

do we force users extension authors to do a require 'jruby'
now, its auto-loaded from kernel which was likely not intentional - should only auto require 'java'
in which case we shall think about NOT auto require 'jruby'
(most exts tend to have an explicit require 'jruby' or use the new load_ext mechanism)
need to propagate load_ext changes to gems we tend to boot on every run (due RGs) :
~~jruby-openssl~~, ~~jruby-readline~~, psych, jar-dependencies
... and get them to propagate releases (so this PR delivers the promised loading speed improvement)

- moved *extend_proxy* to native (although its very useless atm) - *print_class* has been broken - failing with a NoMethodError for quite a long time (all 9K), thus a good excuse to remove

kares · 2018-06-01T14:00:25Z

core/src/main/java/org/jruby/Ruby.java

@@ -1717,7 +1717,6 @@ private void initBuiltins() {
        addLazyBuiltin("java.rb", "java", "org.jruby.javasupport.Java");
        addLazyBuiltin("jruby.rb", "jruby", "org.jruby.ext.jruby.JRubyLibrary");
        addLazyBuiltin("jruby/util.rb", "jruby/util", "org.jruby.ext.jruby.JRubyUtilLibrary");
-        addLazyBuiltin("jruby/type.rb", "jruby/type", "org.jruby.ext.jruby.JRubyTypeLibrary");


org.jruby.ext.jruby.JRubyTypeLibrary class was gone for quite some time - as explained in the commit

kares · 2018-06-01T14:02:39Z

core/src/main/java/org/jruby/RubySignal.java

-            Class realFacadeClass = Class.forName("org.jruby.util.SunSignalFacade");
-            return (SignalFacade)realFacadeClass.newInstance();
-        } catch(Throwable e) {
+            return org.jruby.util.SunSignalFacade.class.newInstance();


somehow ended up hitting this Class.forName spot (at least) twice while 'naively' profiling.
likely a false alarm, but I still did review the class and its signal.rb pieces ...

kares · 2018-06-15T04:28:56Z

// ping @headius and @enebo for a review - so we can start setting up JRuby.load_ext in default gems

enebo

requiring both 'require "java"' or 'require "jruby"' is semantically required even though they both incidentally worked.

You have some code with if and else if not joined with {. Same for catch.

Depending on autoload feels weird to me but we run rails so it is ok by me I guess.

esp. since a JRuby.runtime causes JI loading of org.jruby internals `JRuby.load_ext` to be used by gems that get loaded on boot - RubyGems (e.g. openssl and psych)

... we're no longer exactly matching up with MRI's *prelude.rb* anyway

with MRI ... this also isn't an exact match *gem_prelude.rb* match

…ding JI" This reverts commit 98e97ba.

... we simply generate the Java listener on demand as its to be used

kares · 2018-06-18T13:02:38Z

OK, cleaned up the code-style.

requiring both 'require "java"' or 'require "jruby"' is semantically required even though they both incidentally worked.

at this point than we could remove require 'jruby' from booting up on all JRuby boots?
its unfortunate ... esp. since most of it is covered by the internal JRuby::Util helpers
the downside is mostly parts (user exts) doing JRuby.runtime without the require ...

seems to makes sense as using JRuby internals make the runtime "dirty" e.g. ObjectSpace will find invalid class pieces such as for UNDEF still, its kind of a small "break" in term-of backwards compatibility although most scripts floating around do the explicit require 'jruby' before scripting with internals such as `JRuby.runtime` left-over from #5205 as noted at #5233

kares added 7 commits June 1, 2018 15:00

[refactor] avoid JRuby.runtime usage (on POSIX) when loading kernel.rb

a51f0dc

internal helper to set meta-class (avoid booting JI due self-reflect)

b91cf04

setup a convention to boot psych.rb (similar to MRI) wout loading JI

98e97ba

[refactor] remove (unused) java_utilities.rb

daf4be3

- moved *extend_proxy* to native (although its very useless atm) - *print_class* has been broken - failing with a NoMethodError for quite a long time (all 9K), thus a good excuse to remove

match Thread part from prelude.rb with MRI - likely a bit useless

86ada01

remove (internal) helper that is long overdue by now - no refs around

1262e35

hide include_class as private - its been deprecated long enough

0fbb2b7

kares added this to the JRuby 9.2.1.0 milestone Jun 1, 2018

kares self-assigned this Jun 1, 2018

kares requested review from headius and enebo June 1, 2018 13:55

kares changed the title ~~cleanup jruby boot -> improve performance~~ review (cleanup) boot -> standard JRuby extension loading Jun 1, 2018

kares commented Jun 1, 2018

View reviewed changes

kares mentioned this pull request Jun 11, 2018

Not every class has a superclass? #4528

Closed

enebo approved these changes Jun 15, 2018

View reviewed changes

kares force-pushed the ji-lazy-pp2 branch from 7b8d196 to 4ad609e Compare June 18, 2018 12:34

kares added 12 commits June 18, 2018 14:34

[feat] JRuby way of loading extension - without self reflection

1e26f2a

esp. since a JRuby.runtime causes JI loading of org.jruby internals `JRuby.load_ext` to be used by gems that get loaded on boot - RubyGems (e.g. openssl and psych)

actually, set_meta_class helper is needed on boot - move it under Util

9ad164d

guess sizes (based on gem cmd) to avoid re-hashing (during boot)

6562590

the require 'date.jar' already does load the *DateLibrary*

e17c540

setup require 'securerandom.jar' convention instead of manual loading

d7d3f26

we can avoid org.jruby.platform.Platform load by setting up a const

bc56931

do not pollute Object with Signal details - no such constants on MRI

ce7df52

cleanup and simplify Signal impl details (perform less init work)

0572339

let's not boot a deprecated Thread.exclusive piece in a prelude.rb

b6468f0

... we're no longer exactly matching up with MRI's *prelude.rb* anyway

gem prelude - const_def slightly faster + match require 'rubygems.rb'

339ec5c

with MRI ... this also isn't an exact match *gem_prelude.rb* match

[refactor] avoid load_error.rb loading -> this is also native in MRI

1fad8e3

we already do have pp.rb loaded on **prelude.rb** (same as at MRI's)

683b922

kares added 9 commits June 18, 2018 14:34

redo class_loaded_resources to be (re)usable for jar-dependencies

2cb7e73

[refactor] re-arrange jruby/kernel.rb -> delay some with autoloads

1ed6185

cleanup - JRubyTypeLibrary class hasn't existed for quite some time

f155ae3

Revert "setup a convention to boot psych.rb (similar to MRI) wout loa…

51225b7

…ding JI" This reverts commit 98e97ba.

GC part already loaded from native -> at least delay JI loading on use

c3248dd

... we simply generate the Java listener on demand as its to be used

[refactor] minor - no need to copy-array for RubyArray

5e2085a

[refactor] pass context w internal method; use RubyRange#isExcludeEnd

7bdb76a

[refactor] avoid some (iternal) array cp-ing (redundant) nil fill-ing

967c7aa

[refactor] this is a len > 2 most of the time -> alloc array directly

4ad609e

This was referenced Jun 21, 2018

update jruby-readline to new release #5225

Merged

use newer style of JRuby ext loading when available ruby/psych#365

Closed

kares merged commit 459262a into master Jun 28, 2018

kares deleted the ji-lazy-pp2 branch June 28, 2018 11:17

kares mentioned this pull request Jun 28, 2018

jruby (ext) loading left-overs to consider #5233

Closed

kares mentioned this pull request Oct 23, 2018

only require 'java', users should explicitly require 'jruby' #5381

Merged

kares mentioned this pull request Sep 18, 2020

boot on Windows (still) self reflecting #6396

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

GitHub Sponsors

review (cleanup) boot -> standard JRuby extension loading #5205

review (cleanup) boot -> standard JRuby extension loading #5205

kares commented Jun 1, 2018 •

edited

Loading

kares Jun 1, 2018 •

edited

Loading

kares Jun 1, 2018

kares commented Jun 15, 2018

enebo left a comment

kares commented Jun 18, 2018

review (cleanup) boot -> standard JRuby extension loading #5205

review (cleanup) boot -> standard JRuby extension loading #5205

Conversation

kares commented Jun 1, 2018 • edited Loading

kares Jun 1, 2018 • edited Loading

Choose a reason for hiding this comment

kares Jun 1, 2018

Choose a reason for hiding this comment

kares commented Jun 15, 2018

enebo left a comment

Choose a reason for hiding this comment

kares commented Jun 18, 2018

kares commented Jun 1, 2018 •

edited

Loading

kares Jun 1, 2018 •

edited

Loading