seems like the back-ref isn't stored/getting back properly, trace :

error: java.lang.ClassCastException: org.jruby.RubyNil cannot be cast to org.jruby.RubyMatchData
  org.jruby.RubyString.regexSplit19(RubyString.java:3592)
  org.jruby.RubyString.splitCommon19(RubyString.java:3549)
  org.jruby.RubyString.split19(RubyString.java:3516)
  org.jruby.RubyString$INVOKER$i$split19.call(RubyString$INVOKER$i$split19.gen)
  org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:153)
  split_threads.invokeOther1:split(split_threads.rb:7)

@kares update :)

Note the last line I just added (sorry for not adding that here). Single threaded, this works fine.
Also works fine if using a method defined via simply def instead of the lambda.

Backref is not thread-safe, unfortunately. The problem here is due to the threads all sharing the same backref ref (in this case, the top-level frame). This is documented in other issues, and there's no easy solution.

The simplest workaround is to start each thread in its own method body.

The issue with the ClassCastException can probably be improved; we're obviously expecting the backref to keep pointing at a MatchData from the beginning of the search to the end, and don't re-check it.

This modification makes the script work ok:

def one
  Thread.new do
    puts "starting 1"
    while running do
      break if splittt.call(str).nil?
    end
  end
end
one = self.one

def two
  Thread.new do
    puts "starting 2"
    while running do
      break if splittt.call(str).nil?
    end
  end
end
two = self.two

@headius yea that's what I figured as well, as a matter of fact it even works if I simply make it:

def splittt(strr)
  begin
    res = strr.split("")
    res
  rescue => e
    puts("error")
    puts(e)
  end
end

and call that instead of the lambda. The issue from our perspective is rather, that this change from 1.7.x to 9k breaks a plugin of ours. But if there's no quick fix here, we'll fix that plugin accordingly.

Thanks for looking into this! Should I close here, if this is tracked elsewhere already anyhow?

I think the regression here actually is more of a compatibility improvement :-( It's likely we made improvements in 9k to frame sharing that make it more compatible but less thread-safe.

I have proven that MRI also has this issue, and they have no solution for it either. You want it isolated sometimes, but not always; often the intent is exactly to share the backref with a thread you've just spawned.

https://bugs.ruby-lang.org/issues/12689

My perspective is that each thread should get its own backref frame, cloned from the parent (holding that parent's value but never writing back to it). Unfortunately this isn't a simple fix either; if you have backref sharing in an arbitrary proc, we don't currently specialize anything from that proc for each thread it runs on. That issue is detailed here: #3031

@original-brownbear Yes, that is a simpler change; all that really matters is that the nearest non-block frame doesn't get shared, which you've done here by putting the split call in a separate method.

The bottom line here is that, as we've always known, globals are bad. Even when ostensibly isolated by method frames, we can still run into sharing cases like this.

I'm going to close this, since it doesn't document any new buggy behavior and it's now linked to #3031. I don't know what the solution should be.

Updating this because I did not notice when #6640 was filed that it is actually the same issue (split concurrency problem leading to ClassCastException). This is now FIXED in 9.2.17.0 by #6644.

The general problem is also FIXED for 9.3 by #6647 which avoids ever reading $~ within our core methods that are aware of it. Those methods now only SET the $~ value, avoiding internal races. Users explicitly opting into access of $~, either by accessing the variable or by using methods like last_match that access it, will still need to take care that other threads are not updating it at the same time it is being read.