Good find! Investigating.

SnakeYAML was updated in October, so if that's the culprit it would have first manifested in JRuby 9.1.14.

A quick sampled profile shows only one big hot spot, a method in SnakeYAML: ScannerImpl.scanFlowScalarNonSpaces. On this profile, 650 out of 867 ticks were during this method's execution.

Ah I see you mention it being slow in 9.1.8.0. I'll start looking there for a possible culprit.

I traced it to the Psych update in b0487b4. Continuing.

The only substantive commit in Psych 2.2.3 was ruby/psych@fc55b92, which updated SnakeYAML from 1.17 to 1.18. So we're back to that theory again.

Manually downgrading SnakeYAML from 1.18 to 1.17 does not resolve the issue, so we're back to Psych being the culprit.

Manually downgrading Psych to 2.2.2 does appear to remedy the problem, but I'm still confused because there's really nothing different between 2.2.2 and 2.2.3 except for the SnakeYAML update.

@headius thanks for looking into this and keeping me up-to-date. Hope it gets resolved.

Ok ignore what I said earlier...after changing all references to "1.18" and manually downgrading SnakeYAML, performance does return to normal. I will check if latest SnakeYAML fixes it and if not I'll file an issue with them.

It appears this regression was fixed in a later version of SnakeYAML. Updating Psych to a build of 3.0.2 with SnakeYAML 1.20 returns performance to the original range. I will proceed to update Psych.

great, thank you @headius

The Psych update is on hold until we can get a fix for https://bitbucket.org/asomov/snakeyaml/issues/401/jrubys-psych-library-uses-markgetindex

Hopefully it won't take long.

Ok, so the bad news is that SnakeYAML 1.19 does not appear to fix the issue, so we'll have to wait for 1.21 that fixes the issue I filed so we can release a new Psych.

Short term workaround: you may be able to install an older Psych and use it. If you are using Bundler, adding an older Psych version to e.g. Gemfile may do the trick.

For a more permanent solution, you can replace lib/ruby/stdlib/psych* with the contents of the gem's lib dir. There may be some oddities matching versions with the gemspec, so that may need to be replaced under lib/ruby/gems/shared/specifications/default.

I can confirm that the workaround of downgrading the psych gem works.
Here's what I did.

I added psych version 2.2.2 in my Gemfile.

source 'https://rubygems.org'
gem 'psych', '2.2.2'

And added those two lines at the top of the gist.

require 'rubygems'
require 'bundler/setup'

The output after those changes showed no degradation:

$ jruby -S test_yaml.rb 
9.1.16.0
  0.220000   0.010000   0.230000 (  0.062216)

If you test SnakeYAML 1.21-SNAPSHOT and it is quick enough, we can release it immediately, otherwise it will be released in August 2018.
The performance penalty is most probably caused by more complex Unicode processing. SnakeYAML now properly works with characters above 65536 (Java chars were designed for 2 bytes per char)

@asomov thank you. @headius would you be testing the SnakeYAML 1.21-SNAPSHOT ? thx

@asomov @vassilios Testing now!

@asomov @vassilios The 1.21 version of SnakeYAML does indeed fix the performance issue, and requires only minor tweaks to the Psych code. Ship it! 👍

@headius thank you for testing it!

1.21 released.
Please be aware that now plain scalars are allowed in the flow context.
{foo:bar} -> foo:bar -> null

@asomov Thank you!

Is not yet solved ?

@asomov This issue should not be closed until we have an updated Psych pushed for JRuby 9.1.17. It closed automatically (incorrectly) when I made my commit to Psych.

This is going to get pushed post-9.1.17 unless we can address the failures from updating Psych and its tests. I got both Psych test failures and YAML failures in the existing ruby specs.

@asomov Your comment about plain scalars...that may be the cause of some failures. I'm pushing a JRuby branch shortly for #5142 that should show all the new failures. Some appear to be related to scalars.

I have pushed branch psych_3_update which updates to Psych 3.0.3 that includes the above fixes. We'll need to address the failures before we commit this to a release.

#365 is merged and I believe everything is in place for a Psych release. Waiting to hear from @hsbt about that.

I will release it in this weekend.