Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JRuby threading and/or String#gsub, regexp issue #4552

Open
rob99 opened this issue Mar 29, 2017 · 3 comments
Open

JRuby threading and/or String#gsub, regexp issue #4552

rob99 opened this issue Mar 29, 2017 · 3 comments

Comments

@rob99
Copy link

rob99 commented Mar 29, 2017

jruby 9.1.6.0 (2.3.1) 2016-11-09 0150a76 Java HotSpot(TM) 64-Bit Server VM 24.71-b01 on 1.7.0_71-b14 +jit [mswin32-x86_64]

This came up in a rails 4.2 app but I think I have isolated the problem as having something to do with either JRuby threading or possibly the oniguruma lib.

When I execute this script, it will typically create and clean up threads as expected. Then usually after about 50 or more attempts, one of the threads gets stuck and does not end.

require 'pp'
ct = Thread.current
puts "Main: " + ct.inspect
1000.times do |a|
  puts "Attempt #{a}"
  1000.times do 
    Thread.new do
      10000.times do
        sql = "SELECT t.* FROM ( SELECT ROW_NUMBER() OVER(ORDER BY [app_users].[id] ASC) AS _row_num, [app_users].* FROM [app_users] WHERE [app_users].[id] = 17 ) AS t WHERE t._row_num BETWEEN 1 AND 1"
        sql.gsub(/N'(?:[^']+|'')+'/m){|m|m.truncate(1000, omission:"[TRUNCATED]'")}
      end
    end
  end

  check_count = 0
  while true
    check_count += 1
    puts "Threads: #{Thread.list.size}"
    break if Thread.list.size == 1
    pp Thread.list
    if check_count > 10
      Thread.list.each do |t|
        pp t.backtrace
        pp t.backtrace_locations
      end
    end
    sleep 1
  end
end

Example output:

<snip>
Attempt 69
Threads: 21
[#<Thread:0x23c7bb5f@gsub.rb:1 sleep>,
 #<Thread:0xdde6c53@gsub.rb:1 run>,
 #<Thread:0x70745955 run>,
 #<Thread:0x599c442 run>]
Threads: 2
[#<Thread:0x70745955 run>, #<Thread:0x599c442 run>]
Threads: 2
[#<Thread:0x70745955 run>, #<Thread:0x599c442 run>]
Threads: 2
[#<Thread:0x70745955 run>, #<Thread:0x599c442 run>]
Threads: 2
[#<Thread:0x70745955 run>, #<Thread:0x599c442 run>]
Threads: 2
[#<Thread:0x70745955 run>, #<Thread:0x599c442 run>]
Threads: 2
[#<Thread:0x70745955 run>, #<Thread:0x599c442 run>]
Threads: 2
[#<Thread:0x70745955 run>, #<Thread:0x599c442 run>]
Threads: 2
[#<Thread:0x70745955 run>, #<Thread:0x599c442 run>]
Threads: 2
[#<Thread:0x70745955 run>, #<Thread:0x599c442 run>]
Threads: 2
[#<Thread:0x70745955 run>, #<Thread:0x599c442 run>]
["gsub.rb:23:in `backtrace'",
 "gsub.rb:23:in `block in gsub.rb'",
 "gsub.rb:22:in `block in gsub.rb'",
 "gsub.rb:4:in `times'",
 "gsub.rb:4:in `<main>'"]
["gsub.rb:24:in `backtrace_locations'",
 "gsub.rb:24:in `block in gsub.rb'",
 "gsub.rb:22:in `block in gsub.rb'",
 "gsub.rb:4:in `times'",
 "gsub.rb:4:in `<main>'"]
[]
[]
Threads: 2
[#<Thread:0x70745955 run>, #<Thread:0x599c442 run>]
["gsub.rb:23:in `backtrace'",
 "gsub.rb:23:in `block in gsub.rb'",
 "gsub.rb:22:in `block in gsub.rb'",
 "gsub.rb:4:in `times'",
 "gsub.rb:4:in `<main>'"]
["gsub.rb:24:in `backtrace_locations'",
 "gsub.rb:24:in `block in gsub.rb'",
 "gsub.rb:22:in `block in gsub.rb'",
 "gsub.rb:4:in `times'",
 "gsub.rb:4:in `<main>'"]
[]
[]
Threads: 2
[#<Thread:0x70745955 run>, #<Thread:0x599c442 run>]
["gsub.rb:23:in `backtrace'",
 "gsub.rb:23:in `block in gsub.rb'",
 "gsub.rb:22:in `block in gsub.rb'",
 "gsub.rb:4:in `times'",
 "gsub.rb:4:in `<main>'"]
["gsub.rb:24:in `backtrace_locations'",
 "gsub.rb:24:in `block in gsub.rb'",
 "gsub.rb:22:in `block in gsub.rb'",
 "gsub.rb:4:in `times'",
 "gsub.rb:4:in `<main>'"]
[]
[]
Threads: 2
[#<Thread:0x70745955 run>, #<Thread:0x599c442 run>]
["gsub.rb:23:in `backtrace'",
 "gsub.rb:23:in `block in gsub.rb'",
 "gsub.rb:22:in `block in gsub.rb'",
 "gsub.rb:4:in `times'",
 "gsub.rb:4:in `<main>'"]
["gsub.rb:24:in `backtrace_locations'",
 "gsub.rb:24:in `block in gsub.rb'",
 "gsub.rb:22:in `block in gsub.rb'",
 "gsub.rb:4:in `times'",
 "gsub.rb:4:in `<main>'"]
[]
[]
Threads: 2
[#<Thread:0x70745955 run>, #<Thread:0x599c442 run>]
["gsub.rb:23:in `backtrace'",
 "gsub.rb:23:in `block in gsub.rb'",
 "gsub.rb:22:in `block in gsub.rb'",
 "gsub.rb:4:in `times'",
 "gsub.rb:4:in `<main>'"]
["gsub.rb:24:in `backtrace_locations'",
 "gsub.rb:24:in `block in gsub.rb'",
 "gsub.rb:22:in `block in gsub.rb'",
 "gsub.rb:4:in `times'",
 "gsub.rb:4:in `<main>'"]
[]
[]
Threads: 2
[#<Thread:0x70745955 run>, #<Thread:0x599c442 run>]
["gsub.rb:23:in `backtrace'",
 "gsub.rb:23:in `block in gsub.rb'",
 "gsub.rb:22:in `block in gsub.rb'",
 "gsub.rb:4:in `times'",
 "gsub.rb:4:in `<main>'"]
["gsub.rb:24:in `backtrace_locations'",
 "gsub.rb:24:in `block in gsub.rb'",
 "gsub.rb:22:in `block in gsub.rb'",
 "gsub.rb:4:in `times'",
 "gsub.rb:4:in `<main>'"]
[]
[]
^C

Interestingly, running with --profile seems to cause the issue on attempt 0. See

https://gist.github.com/rob99/3caede73f6d3f1beaab7d379afd30775#file-thread_profile-txt

@enebo
Copy link
Member

enebo commented Mar 29, 2017

I generated a Java stack dump after we get into this and that one extra left over thread has no Java thread behind it anymore (also it does not seem to have a backtrace either). It is almost as if our cleanup did not remove it from some list?

@rob99
Copy link
Author

rob99 commented Mar 29, 2017

BTW I just noticed my script uses String#truncate in the gsub block - its the Rails extension and I ripped this from my rails app code. Anyway this is never executed because the input string here will not pass the regexp.

In case anyone was wondering...

@headius
Copy link
Member

headius commented Apr 19, 2017

This is a known problem with gsub and the magic $~ variable it updates and reads. MRI has similar issues but they're less common because of the GIL.

If you put logic that calls gsub into its own method body, this will work correctly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants