Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cannot store / retrieve utf8 characters with YAML::Store in 1.7.27 #4767

Closed
duritong opened this issue Aug 30, 2017 · 1 comment
Closed

cannot store / retrieve utf8 characters with YAML::Store in 1.7.27 #4767

duritong opened this issue Aug 30, 2017 · 1 comment
Milestone
JRuby 9.1.12.0

Comments

@duritong
Copy link
Contributor

While debugging an issue with Puppetserver and UTF8 data loaded from YAML (through YAML::Store), I found out that YAML::Store can't read or write UTF8 characters from or to a file. While using standard yaml it works fine.

Environment

  • jruby 1.7.27 (1.9.3p551) 2017-05-11 8cdb01a on OpenJDK 64-Bit Server VM 1.8.0_141-b16 +jit [linux-amd64]
  • Fedora 26 x86_64 - rvm installation

It works using 9.1.12.0, however given the default runtime for Puppetserver is still 1.7.27 -> https://docs.puppet.com/puppetserver/5.0/configuration.html#configuring-the-jruby-version

I tried with this version.

What follows is an easy test case to reproduce the problem.

Reproduce

$ cat test.rb 
# -- encoding : utf-8 --

require 'yaml'
s = ::YAML.load_file('/tmp/foo.yaml')

puts "YAML-Read worked: " + s['wrong']
s['works'] = 'Tütü'
File.open('/tmp/foo.yaml','w'){|f| f << s.to_yaml }
s = ::YAML.load_file('/tmp/foo.yaml')
puts "YAML-Store/Read worked: " + s['works']

require 'yaml/store'
s = ::YAML::Store.new('/tmp/foo.yaml')

puts "YAML::Store-Read fails: " + s.transaction { s['wrong'] }
s.transaction { s['fail'] = 'Tütü' }
puts "YAML::Store-Store/Read fails: " + s.transaction { s['fail'] }


$ echo -e "---\nwrong: Tütü\n" > /tmp/foo.yaml
$ rvm use jruby-1.7.27
Using /home/user/.rvm/gems/jruby-1.7.27
$ jruby test.rb 
YAML-Read worked: Tütü
YAML-Store/Read worked: Tütü
Encoding::UndefinedConversionError: ""\xC3"" from ASCII-8BIT to UTF-8
         parse at org/jruby/ext/psych/PsychParser.java:178
  parse_stream at /home/user/.rvm/rubies/jruby-1.7.27/lib/ruby/1.9/psych.rb:375
         parse at /home/user/.rvm/rubies/jruby-1.7.27/lib/ruby/1.9/psych.rb:323
          load at /home/user/.rvm/rubies/jruby-1.7.27/lib/ruby/1.9/psych.rb:250
          load at /home/user/.rvm/rubies/jruby-1.7.27/lib/ruby/1.9/yaml/store.rb:66
     load_data at /home/user/.rvm/rubies/jruby-1.7.27/lib/ruby/1.9/pstore.rb:407
   transaction at /home/user/.rvm/rubies/jruby-1.7.27/lib/ruby/1.9/pstore.rb:322
   synchronize at org/jruby/ext/thread/Mutex.java:149
   transaction at /home/user/.rvm/rubies/jruby-1.7.27/lib/ruby/1.9/pstore.rb:316
        (root) at test.rb:15

$ rvm use jruby-9.1.12.0
Using /home/user/.rvm/gems/jruby-9.1.12.0
$ echo -e "---\nwrong: Tütü\n" > /tmp/foo.yaml
$ jruby test.rb 
YAML-Read worked: Tütü
YAML-Store/Read worked: Tütü
YAML::Store-Read fails: Tütü
YAML::Store-Store/Read fails: Tütü
@duritong duritong mentioned this issue Sep 1, 2017
Closed
@headius
Copy link
Member

headius commented Sep 7, 2017

The fix for #4770 (done manually on master and in ruby/psych#328) is entirely in the Psych library, so whenever they release 3.0.0 it should be possible for you to upgrade it (if it isn't wrapped up inside Puppet).

Since your example is working in 9.1.12.0 we will call this one fixed. We have no plans to release any more JRuby 1.7.x.

Puppet server really needs to update to a JRuby 9k build though. Not sure who we should talk to about that these days.

Relates to and likely fixed by #4784 in all versions that can install it.

@headius headius closed this as completed Sep 7, 2017
@headius headius added this to the JRuby 9.1.12.0 milestone Sep 7, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
JRuby 9.1.12.0
Development

No branches or pull requests
2 participants
@duritong @headius