Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Truffle] Threads may not arrive at safepoint and prevent shutdown #3124

Closed
nirvdrum opened this issue Jul 9, 2015 · 4 comments
Closed

[Truffle] Threads may not arrive at safepoint and prevent shutdown #3124

nirvdrum opened this issue Jul 9, 2015 · 4 comments
Assignees
@eregon
Milestone
Non-Release

Comments

@nirvdrum
Copy link
Contributor

nirvdrum commented Jul 9, 2015

I've recently run into an issue where a set of threads block on a queue. In the meanwhile, the main thread requires a file, which release the GIL, and one of the worker threads then acquires the GIL. I've traced, but unfortunately I can't really figure out what's happening in between. Of the three threads in the following example, 2 end up arriving at the safepoint, while the third is blocked on a ConditionVariable#wait call.

require 'thread'

WORKER_THREAD_COUNT = 2
@q = Queue.new

WORKER_THREAD_COUNT.times { Thread.new(@q) { |q| q.pop } }

require "ostruct"

p "Done"
@nirvdrum nirvdrum changed the title [Truffle] Threads may not arrive at safepoint and prevent shutdown. [Truffle] Threads may not arrive at safepoint and prevent shutdown Jul 9, 2015
@nirvdrum
Copy link
Contributor Author

nirvdrum commented Jul 9, 2015

A stack dump from f491c64:

2015-07-09 13:09:32
Full thread dump Java HotSpot(TM) 64-Bit Server VM (25.45-b02 mixed mode):

"Attach Listener" #14 daemon prio=9 os_prio=0 tid=0x00007fe8e0001000 nid=0x71f1 waiting on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"Ruby Thread@blocker.rb:6" #13 prio=5 os_prio=0 tid=0x00007fe9390f4800 nid=0x332d waiting on condition [0x00007fe90d1d6000]
   java.lang.Thread.State: WAITING (parking)
    at sun.misc.Unsafe.park(Native Method)
    - parking to wait for  <0x00000000f9c5e4f0> (a java.util.concurrent.Phaser$QNode)
    at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
    at java.util.concurrent.Phaser$QNode.block(Phaser.java:1140)
    at java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3320)
    at java.util.concurrent.Phaser.internalAwaitAdvance(Phaser.java:1067)
    at java.util.concurrent.Phaser.arriveAndAwaitAdvance(Phaser.java:690)
    at org.jruby.truffle.runtime.subsystems.SafepointManager.step(SafepointManager.java:107)
    at org.jruby.truffle.runtime.subsystems.SafepointManager.assumptionInvalidated(SafepointManager.java:94)
    at org.jruby.truffle.runtime.subsystems.SafepointManager.poll(SafepointManager.java:73)
    at org.jruby.truffle.runtime.subsystems.SafepointManager.poll(SafepointManager.java:66)
    at org.jruby.truffle.nodes.RubyRootNode.execute(RubyRootNode.java:56)
    at com.oracle.truffle.api.impl.DefaultCallTarget.call(DefaultCallTarget.java:76)
    at com.oracle.truffle.api.impl.DefaultDirectCallNode.call(DefaultDirectCallNode.java:65)
    at org.jruby.truffle.nodes.yield.CachedYieldDispatchNode.dispatchWithSelfAndBlock(CachedYieldDispatchNode.java:61)
    at org.jruby.truffle.nodes.yield.YieldDispatchHeadNode.dispatch(YieldDispatchHeadNode.java:33)
    at org.jruby.truffle.nodes.yield.YieldNode.execute(YieldNode.java:62)
    at org.jruby.truffle.nodes.control.SequenceNode.execute(SequenceNode.java:93)
    at org.jruby.truffle.nodes.exceptions.EnsureNode.execute(EnsureNode.java:34)
    at org.jruby.truffle.nodes.control.SequenceNode.execute(SequenceNode.java:93)
    at org.jruby.truffle.nodes.methods.CatchReturnNode.execute(CatchReturnNode.java:39)
    at org.jruby.truffle.nodes.methods.CatchRetryAsErrorNode.execute(CatchRetryAsErrorNode.java:32)
    at org.jruby.truffle.nodes.methods.ExceptionTranslatingNode.execute(ExceptionTranslatingNode.java:58)
    at org.jruby.truffle.nodes.RubyRootNode.execute(RubyRootNode.java:57)
    at com.oracle.truffle.api.impl.DefaultCallTarget.call(DefaultCallTarget.java:76)
    at com.oracle.truffle.api.impl.DefaultDirectCallNode.call(DefaultDirectCallNode.java:65)
    at org.jruby.truffle.nodes.dispatch.CachedBoxedDispatchNode.executeDispatch(CachedBoxedDispatchNode.java:115)
    at org.jruby.truffle.nodes.dispatch.DispatchHeadNode.dispatch(DispatchHeadNode.java:46)
    at org.jruby.truffle.nodes.dispatch.CallDispatchHeadNode.call(CallDispatchHeadNode.java:34)
    at org.jruby.truffle.nodes.dispatch.RubyCallNode.execute(RubyCallNode.java:149)
    at org.jruby.truffle.nodes.methods.CatchBreakNode.execute(CatchBreakNode.java:43)
    at org.jruby.truffle.nodes.control.SequenceNode.execute(SequenceNode.java:93)
    at org.jruby.truffle.nodes.methods.RedoableNode.execute(RedoableNode.java:34)
    at org.jruby.truffle.nodes.methods.CatchNextNode.execute(CatchNextNode.java:36)
    at org.jruby.truffle.nodes.methods.CatchReturnPlaceholderNode.execute(CatchReturnPlaceholderNode.java:36)
    at org.jruby.truffle.nodes.methods.CatchRetryAsErrorNode.execute(CatchRetryAsErrorNode.java:32)
    at org.jruby.truffle.nodes.RubyRootNode.execute(RubyRootNode.java:57)
    at com.oracle.truffle.api.impl.DefaultCallTarget.call(DefaultCallTarget.java:76)
    at com.oracle.truffle.api.impl.DefaultDirectCallNode.call(DefaultDirectCallNode.java:65)
    at org.jruby.truffle.nodes.yield.CachedYieldDispatchNode.dispatchWithSelfAndBlock(CachedYieldDispatchNode.java:61)
    at org.jruby.truffle.nodes.yield.YieldDispatchHeadNode.dispatch(YieldDispatchHeadNode.java:33)
    at org.jruby.truffle.nodes.yield.YieldNode.execute(YieldNode.java:62)
    at org.jruby.truffle.nodes.control.SequenceNode.execute(SequenceNode.java:93)
    at org.jruby.truffle.nodes.methods.CatchReturnNode.execute(CatchReturnNode.java:39)
    at org.jruby.truffle.nodes.methods.CatchRetryAsErrorNode.execute(CatchRetryAsErrorNode.java:32)
    at org.jruby.truffle.nodes.methods.ExceptionTranslatingNode.execute(ExceptionTranslatingNode.java:58)
    at org.jruby.truffle.nodes.RubyRootNode.execute(RubyRootNode.java:57)
    at com.oracle.truffle.api.impl.DefaultCallTarget.call(DefaultCallTarget.java:76)
    at com.oracle.truffle.api.impl.DefaultDirectCallNode.call(DefaultDirectCallNode.java:65)
    at org.jruby.truffle.nodes.dispatch.CachedBoxedDispatchNode.executeDispatch(CachedBoxedDispatchNode.java:115)
    at org.jruby.truffle.nodes.dispatch.DispatchHeadNode.dispatch(DispatchHeadNode.java:46)
    at org.jruby.truffle.nodes.dispatch.CallDispatchHeadNode.call(CallDispatchHeadNode.java:34)
    at org.jruby.truffle.nodes.dispatch.RubyCallNode.execute(RubyCallNode.java:149)
    at org.jruby.truffle.nodes.methods.CatchBreakNode.execute(CatchBreakNode.java:43)
    at org.jruby.truffle.nodes.control.SequenceNode.execute(SequenceNode.java:93)
    at org.jruby.truffle.nodes.methods.CatchReturnNode.execute(CatchReturnNode.java:39)
    at org.jruby.truffle.nodes.methods.CatchRetryAsErrorNode.execute(CatchRetryAsErrorNode.java:32)
    at org.jruby.truffle.nodes.methods.ExceptionTranslatingNode.execute(ExceptionTranslatingNode.java:58)
    at org.jruby.truffle.nodes.RubyRootNode.execute(RubyRootNode.java:57)
    at com.oracle.truffle.api.impl.DefaultCallTarget.call(DefaultCallTarget.java:76)
    at com.oracle.truffle.api.impl.DefaultDirectCallNode.call(DefaultDirectCallNode.java:65)
    at org.jruby.truffle.nodes.dispatch.CachedBoxedDispatchNode.executeDispatch(CachedBoxedDispatchNode.java:115)
    at org.jruby.truffle.nodes.dispatch.DispatchHeadNode.dispatch(DispatchHeadNode.java:46)
    at org.jruby.truffle.nodes.dispatch.CallDispatchHeadNode.call(CallDispatchHeadNode.java:34)
    at org.jruby.truffle.nodes.dispatch.RubyCallNode.execute(RubyCallNode.java:149)
    at org.jruby.truffle.nodes.control.SequenceNode.execute(SequenceNode.java:93)
    at org.jruby.truffle.nodes.methods.RedoableNode.execute(RedoableNode.java:34)
    at org.jruby.truffle.nodes.methods.CatchNextNode.execute(CatchNextNode.java:36)
    at org.jruby.truffle.nodes.methods.CatchReturnPlaceholderNode.execute(CatchReturnPlaceholderNode.java:36)
    at org.jruby.truffle.nodes.methods.CatchBreakAsProcErrorNode.execute(CatchBreakAsProcErrorNode.java:32)
    at org.jruby.truffle.nodes.methods.CatchRetryAsErrorNode.execute(CatchRetryAsErrorNode.java:32)
    at org.jruby.truffle.nodes.RubyRootNode.execute(RubyRootNode.java:57)
    at com.oracle.truffle.api.impl.DefaultCallTarget.call(DefaultCallTarget.java:76)
    at org.jruby.truffle.runtime.core.RubyProc.rootCall(RubyProc.java:89)
    at org.jruby.truffle.runtime.core.RubyThread$1.run(RubyThread.java:70)
    at org.jruby.truffle.runtime.core.RubyFiber.run(RubyFiber.java:148)
    at org.jruby.truffle.runtime.core.RubyThread.run(RubyThread.java:91)
    at org.jruby.truffle.runtime.core.RubyThread$2.run(RubyThread.java:79)
    at java.lang.Thread.run(Thread.java:745)

"Ruby Thread@blocker.rb:6" #12 prio=5 os_prio=0 tid=0x00007fe9390f2800 nid=0x332c waiting on condition [0x00007fe90d3d6000]
   java.lang.Thread.State: WAITING (parking)
    at sun.misc.Unsafe.park(Native Method)
    - parking to wait for  <0x00000000f9a76770> (a java.util.concurrent.locks.ReentrantLock$NonfairSync)
    at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
    at org.jruby.truffle.nodes.core.ConditionVariableNodes$WaitNode$1.wait(ConditionVariableNodes.java:153)
    at org.jruby.truffle.nodes.core.ConditionVariableNodes$WaitNode$3.block(ConditionVariableNodes.java:203)
    at org.jruby.truffle.nodes.core.ConditionVariableNodes$WaitNode$3.block(ConditionVariableNodes.java:200)
    at org.jruby.truffle.runtime.subsystems.ThreadManager.runOnce(ThreadManager.java:126)
    at org.jruby.truffle.runtime.subsystems.ThreadManager.runUntilResult(ThreadManager.java:104)
    at org.jruby.truffle.nodes.core.ConditionVariableNodes$WaitNode.waitOn(ConditionVariableNodes.java:200)
    at org.jruby.truffle.nodes.core.ConditionVariableNodes$WaitNode.wait(ConditionVariableNodes.java:150)
    at org.jruby.truffle.nodes.core.ConditionVariableNodes$WaitNode.wait(ConditionVariableNodes.java:145)
    at org.jruby.truffle.nodes.core.ConditionVariableNodesFactory$WaitNodeFactory$WaitNodeGen$Wait0Node_.executeRubyBasicObject_(ConditionVariableNodesFactory.java:428)
    at org.jruby.truffle.nodes.core.ConditionVariableNodesFactory$WaitNodeFactory$WaitNodeGen$BaseNode_.acceptAndExecute(ConditionVariableNodesFactory.java:283)
    at com.oracle.truffle.api.dsl.internal.SpecializationNode.uninitialized(SpecializationNode.java:417)
    at org.jruby.truffle.nodes.core.ConditionVariableNodesFactory$WaitNodeFactory$WaitNodeGen$UninitializedNode_.executeRubyBasicObject_(ConditionVariableNodesFactory.java:384)
    at org.jruby.truffle.nodes.core.ConditionVariableNodesFactory$WaitNodeFactory$WaitNodeGen$BaseNode_.execute(ConditionVariableNodesFactory.java:292)
    at org.jruby.truffle.nodes.core.ConditionVariableNodesFactory$WaitNodeFactory$WaitNodeGen.execute(ConditionVariableNodesFactory.java:242)
    at org.jruby.truffle.nodes.control.SequenceNode.execute(SequenceNode.java:93)
    at org.jruby.truffle.nodes.methods.ExceptionTranslatingNode.execute(ExceptionTranslatingNode.java:58)
    at org.jruby.truffle.nodes.RubyRootNode.execute(RubyRootNode.java:57)
    at com.oracle.truffle.api.impl.DefaultCallTarget.call(DefaultCallTarget.java:76)
    at com.oracle.truffle.api.impl.DefaultDirectCallNode.call(DefaultDirectCallNode.java:65)
    at org.jruby.truffle.nodes.dispatch.CachedBoxedDispatchNode.executeDispatch(CachedBoxedDispatchNode.java:115)
    at org.jruby.truffle.nodes.dispatch.UnresolvedDispatchNode.executeDispatch(UnresolvedDispatchNode.java:108)
    at org.jruby.truffle.nodes.dispatch.DispatchHeadNode.dispatch(DispatchHeadNode.java:46)
    at org.jruby.truffle.nodes.dispatch.CallDispatchHeadNode.call(CallDispatchHeadNode.java:34)
    at org.jruby.truffle.nodes.dispatch.RubyCallNode.execute(RubyCallNode.java:149)
    at org.jruby.truffle.nodes.control.SequenceNode.execute(SequenceNode.java:93)
    at org.jruby.truffle.nodes.exceptions.EnsureNode.execute(EnsureNode.java:34)
    at org.jruby.truffle.nodes.control.SequenceNode.execute(SequenceNode.java:93)
    at org.jruby.truffle.nodes.control.IfNode.execute(IfNode.java:56)
    at org.jruby.truffle.nodes.control.SequenceNode.execute(SequenceNode.java:93)
    at org.jruby.truffle.nodes.control.IfNode.execute(IfNode.java:53)
    at org.jruby.truffle.nodes.control.SequenceNode.execute(SequenceNode.java:93)
    at org.jruby.truffle.nodes.control.WhileNode$WhileRepeatingNode.executeRepeating(WhileNode.java:83)
    at com.oracle.truffle.api.impl.DefaultLoopNode.executeLoop(DefaultLoopNode.java:45)
    at org.jruby.truffle.nodes.control.WhileNode.execute(WhileNode.java:47)
    at org.jruby.truffle.nodes.methods.CatchBreakNode.execute(CatchBreakNode.java:43)
    at org.jruby.truffle.nodes.control.SequenceNode.execute(SequenceNode.java:93)
    at org.jruby.truffle.nodes.methods.RedoableNode.execute(RedoableNode.java:34)
    at org.jruby.truffle.nodes.methods.CatchNextNode.execute(CatchNextNode.java:36)
    at org.jruby.truffle.nodes.methods.CatchReturnPlaceholderNode.execute(CatchReturnPlaceholderNode.java:36)
    at org.jruby.truffle.nodes.methods.CatchRetryAsErrorNode.execute(CatchRetryAsErrorNode.java:32)
    at org.jruby.truffle.nodes.RubyRootNode.execute(RubyRootNode.java:57)
    at com.oracle.truffle.api.impl.DefaultCallTarget.call(DefaultCallTarget.java:76)
    at com.oracle.truffle.api.impl.DefaultDirectCallNode.call(DefaultDirectCallNode.java:65)
    at org.jruby.truffle.nodes.yield.CachedYieldDispatchNode.dispatchWithSelfAndBlock(CachedYieldDispatchNode.java:61)
    at org.jruby.truffle.nodes.yield.UninitializedYieldDispatchNode.dispatchWithSelfAndBlock(UninitializedYieldDispatchNode.java:70)
    at org.jruby.truffle.nodes.yield.UninitializedYieldDispatchNode.dispatchWithSelfAndBlock(UninitializedYieldDispatchNode.java:70)
    at org.jruby.truffle.nodes.yield.YieldDispatchHeadNode.dispatch(YieldDispatchHeadNode.java:33)
    at org.jruby.truffle.nodes.yield.YieldNode.execute(YieldNode.java:62)
    at org.jruby.truffle.nodes.control.SequenceNode.execute(SequenceNode.java:93)
    at org.jruby.truffle.nodes.exceptions.EnsureNode.execute(EnsureNode.java:34)
    at org.jruby.truffle.nodes.control.SequenceNode.execute(SequenceNode.java:93)
    at org.jruby.truffle.nodes.methods.CatchReturnNode.execute(CatchReturnNode.java:39)
    at org.jruby.truffle.nodes.methods.CatchRetryAsErrorNode.execute(CatchRetryAsErrorNode.java:32)
    at org.jruby.truffle.nodes.methods.ExceptionTranslatingNode.execute(ExceptionTranslatingNode.java:58)
    at org.jruby.truffle.nodes.RubyRootNode.execute(RubyRootNode.java:57)
    at com.oracle.truffle.api.impl.DefaultCallTarget.call(DefaultCallTarget.java:76)
    at com.oracle.truffle.api.impl.DefaultDirectCallNode.call(DefaultDirectCallNode.java:65)
    at org.jruby.truffle.nodes.dispatch.CachedBoxedDispatchNode.executeDispatch(CachedBoxedDispatchNode.java:115)
    at org.jruby.truffle.nodes.dispatch.UnresolvedDispatchNode.executeDispatch(UnresolvedDispatchNode.java:108)
    at org.jruby.truffle.nodes.dispatch.DispatchHeadNode.dispatch(DispatchHeadNode.java:46)
    at org.jruby.truffle.nodes.dispatch.CallDispatchHeadNode.call(CallDispatchHeadNode.java:34)
    at org.jruby.truffle.nodes.dispatch.RubyCallNode.execute(RubyCallNode.java:149)
    at org.jruby.truffle.nodes.methods.CatchBreakNode.execute(CatchBreakNode.java:43)
    at org.jruby.truffle.nodes.control.SequenceNode.execute(SequenceNode.java:93)
    at org.jruby.truffle.nodes.methods.RedoableNode.execute(RedoableNode.java:34)
    at org.jruby.truffle.nodes.methods.CatchNextNode.execute(CatchNextNode.java:36)
    at org.jruby.truffle.nodes.methods.CatchReturnPlaceholderNode.execute(CatchReturnPlaceholderNode.java:36)
    at org.jruby.truffle.nodes.methods.CatchRetryAsErrorNode.execute(CatchRetryAsErrorNode.java:32)
    at org.jruby.truffle.nodes.RubyRootNode.execute(RubyRootNode.java:57)
    at com.oracle.truffle.api.impl.DefaultCallTarget.call(DefaultCallTarget.java:76)
    at com.oracle.truffle.api.impl.DefaultDirectCallNode.call(DefaultDirectCallNode.java:65)
    at org.jruby.truffle.nodes.yield.CachedYieldDispatchNode.dispatchWithSelfAndBlock(CachedYieldDispatchNode.java:61)
    at org.jruby.truffle.nodes.yield.UninitializedYieldDispatchNode.dispatchWithSelfAndBlock(UninitializedYieldDispatchNode.java:70)
    at org.jruby.truffle.nodes.yield.UninitializedYieldDispatchNode.dispatchWithSelfAndBlock(UninitializedYieldDispatchNode.java:70)
    at org.jruby.truffle.nodes.yield.YieldDispatchHeadNode.dispatch(YieldDispatchHeadNode.java:33)
    at org.jruby.truffle.nodes.yield.YieldNode.execute(YieldNode.java:62)
    at org.jruby.truffle.nodes.control.SequenceNode.execute(SequenceNode.java:93)
    at org.jruby.truffle.nodes.methods.CatchReturnNode.execute(CatchReturnNode.java:39)
    at org.jruby.truffle.nodes.methods.CatchRetryAsErrorNode.execute(CatchRetryAsErrorNode.java:32)
    at org.jruby.truffle.nodes.methods.ExceptionTranslatingNode.execute(ExceptionTranslatingNode.java:58)
    at org.jruby.truffle.nodes.RubyRootNode.execute(RubyRootNode.java:57)
    at com.oracle.truffle.api.impl.DefaultCallTarget.call(DefaultCallTarget.java:76)
    at com.oracle.truffle.api.impl.DefaultDirectCallNode.call(DefaultDirectCallNode.java:65)
    at org.jruby.truffle.nodes.dispatch.CachedBoxedDispatchNode.executeDispatch(CachedBoxedDispatchNode.java:115)
    at org.jruby.truffle.nodes.dispatch.UnresolvedDispatchNode.executeDispatch(UnresolvedDispatchNode.java:108)
    at org.jruby.truffle.nodes.dispatch.DispatchHeadNode.dispatch(DispatchHeadNode.java:46)
    at org.jruby.truffle.nodes.dispatch.CallDispatchHeadNode.call(CallDispatchHeadNode.java:34)
    at org.jruby.truffle.nodes.dispatch.RubyCallNode.execute(RubyCallNode.java:149)
    at org.jruby.truffle.nodes.methods.CatchBreakNode.execute(CatchBreakNode.java:43)
    at org.jruby.truffle.nodes.control.SequenceNode.execute(SequenceNode.java:93)
    at org.jruby.truffle.nodes.methods.CatchReturnNode.execute(CatchReturnNode.java:39)
    at org.jruby.truffle.nodes.methods.CatchRetryAsErrorNode.execute(CatchRetryAsErrorNode.java:32)
    at org.jruby.truffle.nodes.methods.ExceptionTranslatingNode.execute(ExceptionTranslatingNode.java:58)
    at org.jruby.truffle.nodes.RubyRootNode.execute(RubyRootNode.java:57)
    at com.oracle.truffle.api.impl.DefaultCallTarget.call(DefaultCallTarget.java:76)
    at com.oracle.truffle.api.impl.DefaultDirectCallNode.call(DefaultDirectCallNode.java:65)
    at org.jruby.truffle.nodes.dispatch.CachedBoxedDispatchNode.executeDispatch(CachedBoxedDispatchNode.java:115)
    at org.jruby.truffle.nodes.dispatch.UnresolvedDispatchNode.executeDispatch(UnresolvedDispatchNode.java:108)
    at org.jruby.truffle.nodes.dispatch.DispatchHeadNode.dispatch(DispatchHeadNode.java:46)
    at org.jruby.truffle.nodes.dispatch.CallDispatchHeadNode.call(CallDispatchHeadNode.java:34)
    at org.jruby.truffle.nodes.dispatch.RubyCallNode.execute(RubyCallNode.java:149)
    at org.jruby.truffle.nodes.control.SequenceNode.execute(SequenceNode.java:93)
    at org.jruby.truffle.nodes.methods.RedoableNode.execute(RedoableNode.java:34)
    at org.jruby.truffle.nodes.methods.CatchNextNode.execute(CatchNextNode.java:36)
    at org.jruby.truffle.nodes.methods.CatchReturnPlaceholderNode.execute(CatchReturnPlaceholderNode.java:36)
    at org.jruby.truffle.nodes.methods.CatchBreakAsProcErrorNode.execute(CatchBreakAsProcErrorNode.java:32)
    at org.jruby.truffle.nodes.methods.CatchRetryAsErrorNode.execute(CatchRetryAsErrorNode.java:32)
    at org.jruby.truffle.nodes.RubyRootNode.execute(RubyRootNode.java:57)
    at com.oracle.truffle.api.impl.DefaultCallTarget.call(DefaultCallTarget.java:76)
    at org.jruby.truffle.runtime.core.RubyProc.rootCall(RubyProc.java:89)
    at org.jruby.truffle.runtime.core.RubyThread$1.run(RubyThread.java:70)
    at org.jruby.truffle.runtime.core.RubyFiber.run(RubyFiber.java:148)
    at org.jruby.truffle.runtime.core.RubyThread.run(RubyThread.java:91)
    at org.jruby.truffle.runtime.core.RubyThread$2.run(RubyThread.java:79)
    at java.lang.Thread.run(Thread.java:745)

"Service Thread" #9 daemon prio=9 os_prio=0 tid=0x00007fe9380d9800 nid=0x3314 runnable [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"C1 CompilerThread3" #8 daemon prio=9 os_prio=0 tid=0x00007fe9380cc000 nid=0x3313 waiting on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"C2 CompilerThread2" #7 daemon prio=9 os_prio=0 tid=0x00007fe9380ca000 nid=0x3312 waiting on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"C2 CompilerThread1" #6 daemon prio=9 os_prio=0 tid=0x00007fe9380c8000 nid=0x3311 waiting on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"C2 CompilerThread0" #5 daemon prio=9 os_prio=0 tid=0x00007fe9380c5800 nid=0x3310 waiting on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"Signal Dispatcher" #4 daemon prio=9 os_prio=0 tid=0x00007fe9380c4000 nid=0x330f runnable [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"Finalizer" #3 daemon prio=8 os_prio=0 tid=0x00007fe93808b800 nid=0x330d in Object.wait() [0x00007fe920779000]
   java.lang.Thread.State: WAITING (on object monitor)
    at java.lang.Object.wait(Native Method)
    at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:143)
    - locked <0x00000000e0c03400> (a java.lang.ref.ReferenceQueue$Lock)
    at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:164)
    at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:209)

"Reference Handler" #2 daemon prio=10 os_prio=0 tid=0x00007fe938089800 nid=0x330c in Object.wait() [0x00007fe92097a000]
   java.lang.Thread.State: WAITING (on object monitor)
    at java.lang.Object.wait(Native Method)
    at java.lang.Object.wait(Object.java:502)
    at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:157)
    - locked <0x00000000e0c0af10> (a java.lang.ref.Reference$Lock)

"main" #1 prio=5 os_prio=0 tid=0x00007fe93800a000 nid=0x3300 waiting on condition [0x00007fe93d923000]
   java.lang.Thread.State: WAITING (parking)
    at sun.misc.Unsafe.park(Native Method)
    - parking to wait for  <0x00000000f9dccff0> (a java.util.concurrent.Phaser$QNode)
    at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
    at java.util.concurrent.Phaser$QNode.block(Phaser.java:1140)
    at java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3320)
    at java.util.concurrent.Phaser.internalAwaitAdvance(Phaser.java:1067)
    at java.util.concurrent.Phaser.arriveAndAwaitAdvance(Phaser.java:690)
    at org.jruby.truffle.runtime.subsystems.SafepointManager.step(SafepointManager.java:107)
    at org.jruby.truffle.runtime.subsystems.SafepointManager.assumptionInvalidated(SafepointManager.java:94)
    at org.jruby.truffle.runtime.subsystems.SafepointManager.pauseAllThreadsAndExecute(SafepointManager.java:182)
    at org.jruby.truffle.runtime.subsystems.SafepointManager.pauseAllThreadsAndExecute(SafepointManager.java:139)
    at org.jruby.truffle.runtime.subsystems.ThreadManager.killOtherThreads(ThreadManager.java:165)
    at org.jruby.truffle.runtime.subsystems.ThreadManager.shutdown(ThreadManager.java:154)
    at org.jruby.truffle.runtime.RubyContext.innerShutdown(RubyContext.java:379)
    at org.jruby.truffle.runtime.RubyContext.shutdown(RubyContext.java:647)
    at org.jruby.Ruby.shutdownTruffleContextIfRunning(Ruby.java:919)
    - locked <0x00000000e0c16a90> (a java.lang.Object)
    at org.jruby.Main.doRunFromMain(Main.java:403)
    at org.jruby.Main.internalRun(Main.java:296)
    at org.jruby.Main.run(Main.java:225)
    at org.jruby.Main.main(Main.java:197)

"VM Thread" os_prio=0 tid=0x00007fe938084800 nid=0x330b runnable 

"GC task thread#0 (ParallelGC)" os_prio=0 tid=0x00007fe93801f800 nid=0x3301 runnable 

"GC task thread#1 (ParallelGC)" os_prio=0 tid=0x00007fe938021000 nid=0x3302 runnable 

"GC task thread#2 (ParallelGC)" os_prio=0 tid=0x00007fe938023000 nid=0x3303 runnable 

"GC task thread#3 (ParallelGC)" os_prio=0 tid=0x00007fe938024800 nid=0x3304 runnable 

"GC task thread#4 (ParallelGC)" os_prio=0 tid=0x00007fe938026800 nid=0x3305 runnable 

"GC task thread#5 (ParallelGC)" os_prio=0 tid=0x00007fe938028000 nid=0x3306 runnable 

"GC task thread#6 (ParallelGC)" os_prio=0 tid=0x00007fe93802a000 nid=0x3307 runnable 

"GC task thread#7 (ParallelGC)" os_prio=0 tid=0x00007fe93802b800 nid=0x3308 runnable 

"GC task thread#8 (ParallelGC)" os_prio=0 tid=0x00007fe93802d800 nid=0x3309 runnable 

"GC task thread#9 (ParallelGC)" os_prio=0 tid=0x00007fe93802f000 nid=0x330a runnable 

"VM Periodic Task Thread" os_prio=0 tid=0x00007fe9380dc000 nid=0x3315 waiting on condition 

JNI global references: 35

@nirvdrum
Copy link
Contributor Author

nirvdrum commented Jul 9, 2015

It looks like the Condition in ConditionVariable#wait (from

jruby/truffle/src/main/java/org/jruby/truffle/nodes/core/ConditionVariableNodes.java

Line 153 in 452dfcf

condition.await();
) isn't interrupting, preventing the thread from hitting the safepoint.

@eregon
Copy link
Member

eregon commented Jul 10, 2015

OK, so one problem here is standard debugging tools do not show ReentrantLocks, which th queue ones are likely locked here but we do not see it.
And since Condition.await MUST re-acquire the lock when it leaves (its post-condition), even if it is interrupted, it never manages to do that because other threads are in the safepoint.

Fix: re-acquire that lock interruptibly, so implement our own Condition variable or use the Ruby code.

@eregon eregon closed this as completed in 3faca64 Jul 10, 2015
@eregon
Copy link
Member

eregon commented Jul 10, 2015

Should be fixed now.
The Ruby implementation might have some problems however, and another implementation might be required.
But the focus should be on Queue, since there is a nice potential to optimize it compared to the current version.

@enebo enebo added this to the truffle-dev milestone Jul 22, 2015
@enebo enebo added this to the Non-Release milestone Dec 7, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@eregon eregon

Labels
None yet
Projects
None yet
Milestone
Non-Release
Development

No branches or pull requests
3 participants
@nirvdrum @enebo @eregon