-
-
Notifications
You must be signed in to change notification settings - Fork 925
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
excessive loading of same InterfaceImpl #5023
Comments
believe they're cached to some extend. |
I half wondered if vert.x isolates via classpaths and perhaps confuses a cache as needing to regenerate something? |
If I can find time, I'll try modify jruby's source to print the interfaces, that should help me to figure out which instance is causing this. Right now I can hardly locate it by any means, or would you have a another idea? On the upside the |
Could you expand a bit on that, I don't understand what you mean. |
Ignore that comment. I was wondering how we generated real classes relative to classloaders. in order to load code all generated types must be reachable from a classloader. If there was some scenario where an execution environment stood up new CLs then the generated types would never be in the right CL and we would generate the thing again (old one eventually GCing). I cannot concretely explain that so it was just an off the cuff comment. |
Here is a reproducer, actually seems to be more of a diagnostic issue, though I still wonder if it could be cached. require './guava-20.0.jar'
# https://github.com/google/guava/wiki/ListenableFutureExplained
# Guava ListenableFuture used by asnyc cassandra API
module Guava
java_import 'com.google.common.util.concurrent.Futures'
java_import 'com.google.common.util.concurrent.FutureCallback'
java_import 'com.google.common.util.concurrent.SettableFuture'
end
$sum = 0
class FutureCallback < Guava::FutureCallback.class
def onSuccess(v)
$sum += v
end
def onFailure(e)
puts e
exit(1)
end
end
class Dummy
end
1000.times do |i|
fut = Guava::SettableFuture.create
Guava::Futures.addCallback(
fut,
# argument dynamically adapted to the Guava::FutureCallback interface on every call
FutureCallback.new(),
->(action) { action.run }
)
fut.set(i)
end
ok = $sum == 1000 * 999 / 2
puts ok ? 'OK' : 'FAIL'
exit(ok ? 0 : 1) So the problem is that the java method accepts any ruby class as interface argument and only binds the method at call time. This is prolly done to support things like MethodMissing, but it's easy to run into and hard to diagnose. The reason why it's
is because I misunderstood the error message for
Searching the net hinted towards
, but that is intended for generating a java class, not a ruby class. The only correct way to implement a java interface in a ruby class is
. Two actions that would improve the status quo.
instead of the
|
there's too many info for me, yet not sure about the reproducer, you expect the 'action.run' impl to generate a cached Java class, right? |
Not at all, it's about the FutureCallback ruby class being dynamically (ad-hoc) adapted to the Guava::FutureCallback interface on every single method invokation. That seems to be an intentional feature of JRuby, kind of duck-typing for interfaces. |
Hope that helps you as TL;DR @kares |
If you want to implement FutureCallback, just do class Blah
include Guava::FutureCallback
...
end The implementation is generated at (I believe) construction time and not regenerated for future objects. The pattern you use in your reproduction may be regenerating the impl class because it's doing it at the point where you pass the object, rather than in the object's natural class. This indicates a problem with the overhead of dynamically implementing interfaces in an arbitrary object. Doing it a bit more statically by including the interface into a concrete class should work around it. @MartinNowak Please try the code above for your FutureCallback impl and let us know how it looks. |
Yes, that's what I figured out after a lot of debugging (see #5023 (comment)). |
I would personally be fine with disabling the feature altogether. I think this is the first time I've seen it used in the wild. I'm not sure I realized it even existed! However, I think implemented properly it could still be efficient and not leak references as in #4968. Ideally, each interface should only ever need a single class generated for these cases, since they all do the same thing: dynamically dispatch all methods from the interface. Anyone else have an opinion on whether to keep and fix the feature or just remove it? |
@kares @byteit101 Ok we need to revisit this. Are we doing a better job of caching the temporary interface impls generated when an object dynamically implements an interface when calling Java? We should also consider deprecating or removing the feature that allows you to extend an interface with the |
Just encountered this in production. The fix for us was to implement the required Java interface using class MyResponse
include javax.servlet.http.HttpServletResponse
...
end |
Environment
Provide at least:
jruby -J-Xms1g -J-Xmx1g -J-XX:+UseG1GC -J-XX:MetaspaceSize=192M -Xcompile.invokedynamic=true
Other relevant info you may wish to add:
Expected Behavior
Attached a verbose class loading log, digged into this when seeing 2+M loaded classes in JConsole, most being collected again, ~50K classes remain resident.
iface_impl.log
With
-Xinterfaces.useProxy=true
I'm only seeing ~13K resident classes and ~750MiB less memory being used (guess that's Metaspace), so that seems like a viable workaround for now.The text was updated successfully, but these errors were encountered: