Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to use JRuby with google app engine due to use of sun.misc.Unsafe #2304

Closed
jaydonnell opened this issue Dec 10, 2014 · 10 comments
Closed

Comments

@jaydonnell
Copy link

Google App Engine does not allow use of sun.misc.Unsafe, and recent versions of JRuby appear to depend on it. Is there (or can there be) a workaround, or will JRuby be unusable on app engine going forward?

@headius
Copy link
Member

headius commented Dec 10, 2014

JRuby should be able to run without sun.misc.Unsafe, falling back on more-safe-but-less-fast code. Can you investigate where we're using it unconditionally?

@jaydonnell
Copy link
Author

Absolutely, I'll dig in when i get a little bit of free time. In the mean time, here is the stack trace in case it's immediately obvious to someone else where the issue is. This occurred on a freshly created rails app, used warbler to make a war, then tried to run the exploded war with the local app engine dev server.

I was using jruby 1.7.16.2, but tried a few other recent versions.

java.lang.NoClassDefFoundError: sun.misc.Unsafe is a restricted class. Please see the Google  App Engine developer's guide for more details.
    at com.google.appengine.tools.development.agent.runtime.Runtime.reject(Runtime.java:51)
    at org.jruby.RubyEncoding.<clinit>(RubyEncoding.java:204)
    at org.jruby.RubyThread$Status.<init>(RubyThread.java:144)
    at org.jruby.RubyThread$Status.<clinit>(RubyThread.java:139)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:601)
    at java.lang.Class.getEnumConstantsShared(Class.java:2957)
    at java.lang.System$2.getEnumConstantsShared(System.java:1185)
    at java.util.EnumMap.getKeyUniverse(EnumMap.java:749)
    at java.util.EnumMap.<init>(EnumMap.java:137)
    at org.jruby.Ruby.<init>(Ruby.java:4924)
    at org.jruby.Ruby.newInstance(Ruby.java:329)
    at org.jruby.rack.DefaultRackApplicationFactory.newRuntime(DefaultRackApplicationFactory.java:334)
    at org.jruby.rack.DefaultRackApplicationFactory$RackApplicationImpl.<init>(DefaultRackApplicationFactory.java:430)
    at org.jruby.rack.DefaultRackApplicationFactory.createApplication(DefaultRackApplicationFactory.java:418)
    at org.jruby.rack.DefaultRackApplicationFactory.newApplication(DefaultRackApplicationFactory.java:98)
    at org.jruby.rack.DefaultRackApplicationFactory.getApplication(DefaultRackApplicationFactory.java:112)
    at org.jruby.rack.PoolingRackApplicationFactory.createApplication(PoolingRackApplicationFactory.java:341)
    at org.jruby.rack.PoolingRackApplicationFactory.getApplicationImpl(PoolingRackApplicationFactory.java:181)
    at org.jruby.rack.RackApplicationFactoryDecorator.getApplication(RackApplicationFactoryDecorator.java:137)
    at org.jruby.rack.DefaultRackDispatcher.getApplication(DefaultRackDispatcher.java:37)
    at org.jruby.rack.AbstractRackDispatcher.process(AbstractRackDispatcher.java:32)
    at org.jruby.rack.AbstractFilter.doFilter(AbstractFilter.java:66)
    at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
    at com.google.appengine.api.socket.dev.DevSocketFilter.doFilter(DevSocketFilter.java:74)
    at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
    at com.google.appengine.tools.development.ResponseRewriterFilter.doFilter(ResponseRewriterFilter.java:127)
    at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
    at com.google.appengine.tools.development.HeaderVerificationFilter.doFilter(HeaderVerificationFilter.java:34)
    at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
    at com.google.appengine.api.blobstore.dev.ServeBlobFilter.doFilter(ServeBlobFilter.java:63)
    at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
    at com.google.apphosting.utils.servlet.TransactionCleanupFilter.doFilter(TransactionCleanupFilter.java:43)
    at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
    at com.google.appengine.tools.development.StaticFileFilter.doFilter(StaticFileFilter.java:125)
    at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
    at com.google.appengine.tools.development.DevAppServerModulesFilter.doDirectRequest(DevAppServerModulesFilter.java:366)
    at com.google.appengine.tools.development.DevAppServerModulesFilter.doDirectModuleRequest(DevAppServerModulesFilter.java:349)
    at com.google.appengine.tools.development.DevAppServerModulesFilter.doFilter(DevAppServerModulesFilter.java:116)
    at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
    at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:388)
    at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
    at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
    at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
    at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:418)
    at com.google.appengine.tools.development.DevAppEngineWebAppContext.handle(DevAppEngineWebAppContext.java:98)
    at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
    at com.google.appengine.tools.development.JettyContainerService$ApiProxyHandler.handle(JettyContainerService.java:491)
    at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
    at org.mortbay.jetty.Server.handle(Server.java:326)
    at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
    at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:923)
    at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:547)
    at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
    at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
    at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:409)
    at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)

@mdavidn
Copy link

mdavidn commented Dec 11, 2014

The feature detection logic itself triggers the rejection. sun.misc.Unsafe exists and loads, but application code must not call its methods.

I'm searching for techniques to detect Google's sandbox. The best option I've found so far is the com.google.appengine.runtime.version system property. If it exists, set UnsafeHolder.U to null?

@headius
Copy link
Member

headius commented Dec 12, 2014

I think the right fix here might be to expand the Exception catch around loading Unsafe; the reason this is percolating out and not falling back on non-Unsafe logic is because GAE is triggering NoClassDefFoundError (not catchable with "Exception") rather than a more typical ClassNotFoundException. I'm not sure why they decided to use an Error, since those are usually reserved for unrecoverable problems (OutOfMemoryError, StackOverflowError, NoClassDefFoundError when a statically-referenced class can't load, etc).

There is a small risk that catching Throwable might mask other critical errors, so I think we should add an additional catch of NoClassDefFoundError rather than widening the existing catch.

@headius
Copy link
Member

headius commented Dec 12, 2014

Here's a trivial patch for you to try, @jaydonnell:

diff --git a/core/src/main/java/org/jruby/util/unsafe/UnsafeHolder.java b/core/src/main/java/org/jruby/util/unsafe/UnsafeHolder.java
index 652ff2d..afb1274 100644
--- a/core/src/main/java/org/jruby/util/unsafe/UnsafeHolder.java
+++ b/core/src/main/java/org/jruby/util/unsafe/UnsafeHolder.java
@@ -50,6 +50,9 @@ public final class UnsafeHolder {
             return (sun.misc.Unsafe) f.get(null);
         } catch (Exception e) {
             return null;
+        } catch (NoClassDefFoundError ncdfe) {
+            // Google AppEngine raises NCDFE for Unsafe rather than CNFE
+            return null;
         }
     }

Clone JRuby, see BUILDING.md.

@headius
Copy link
Member

headius commented Dec 12, 2014

@mdavidn Checking the property may be a reasonable fix, but it feels a bit hacky to me. I also don't like the idea of adding GAE-specific checks in our codebase, since there may be similar checks for other JVMs and clouds in the future...but we do already check for J9 (IBM's JVM) in places, so my point may be moot.

For now I think we'll stick to the exception-based fallback and just make sure it handles NCDFE.

@headius
Copy link
Member

headius commented Dec 12, 2014

I've pushed my patch to jruby-1_7 branch, since it should cover the NCDFE issue, and since all our Unsafe access goes through this same path. Give it a whirl from HEAD or wait for 1.7.18 in a few days.

@headius headius closed this as completed Dec 12, 2014
@headius headius added this to the JRuby 1.7.18 milestone Dec 12, 2014
@jaydonnell
Copy link
Author

Hey @headius, this didn't work. The error gets thrown before the loadUnsafe() is executed so the catch for NoClassDefFoundError needs to be at every usage of UnsafeHolder which is obviously a bad solution. My Java is not great and I haven't been able to come up with a good solution. Any thoughts?

@hakanai
Copy link

hakanai commented Dec 28, 2014

Ah, so maybe that's why it was a NoClassDefFoundError - the class itself has fields and methods using Unsafe and Google's magic runtime is pretending it doesn't exist. So the code never even gets to the Class.forName call... maybe if it did get there, it would have thrown ClassNotFoundException instead [100% speculation].

What I would have done is to make my own Unsafe interface with a RealUnsafe and DummyUnsafe implementations, then in UnsafeHolder, loadUnsafe() would return one of those instead. As an added bonus, you wouldn't need all these "U == null" checks throughout the code.

@jaydonnell
Copy link
Author

@trejkaz yeah, i went that route (DummyUnsafe) interface, and it got by this particular error. Unfortunately I ran into a number of other issues trying to run on app engine. At this point I'm debating whether it's worth the time to continue going down this path of trying to get jruby to work, or if I should give up on jruby on app engine altogether.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants