Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Encoding::ConverterNotFoundError when using Encoding.default_internal #3647

Closed
rykov opened this issue Feb 5, 2016 · 11 comments
Closed

Encoding::ConverterNotFoundError when using Encoding.default_internal #3647

rykov opened this issue Feb 5, 2016 · 11 comments
Labels
parser
Milestone
JRuby 9.1.0.0

Comments

@rykov
Copy link

rykov commented Feb 5, 2016

jRuby versions affected: 9.0.4.0, 9.0.5.0 and possibly earlier ones that I didn't test.

jRuby throws an Encoding::ConverterNotFoundError error when trying to open a BOM:UTF-8 file after setting Encoding.default_internal. This does not mirror the behavior of MRI.

I was able to narrow it down to the following reproduction snippet:

# example.rb
Encoding.default_internal = 'utf-8'
File.open('empty-file', "rb:bom|utf-8") { |f| f.read }
puts "OK!"

Here are the erroneous results:

$ touch empty-file
$ rbenv shell jruby-9.0.4.0 
$ jruby ./example.rb 
Encoding::ConverterNotFoundError: code converter not found (UTF-8 to UTF-8)
...
$ rbenv shell jruby-9.0.5.0 
$ jruby ./example.rb 
Encoding::ConverterNotFoundError: code converter not found (UTF-8 to UTF-8)
...

And here's the same script working with MRI 2.2.4, 2.3.0, and jRuby 1.7.24:

$ rbenv shell 2.2.4 
$ ruby ./example.rb 
OK!
$ rbenv shell 2.3.0 
$ ruby ./example.rb 
OK!
$ rbenv shell jruby-1.7.24 
$ jruby ./example.rb 
OK!

My environment:

$ uname -a
Darwin MBA.local 15.3.0 Darwin Kernel Version 15.3.0: Thu Dec 10 18:40:58 PST 2015; root:xnu-3248.30.4~1/RELEASE_X86_64 x86_64
$ java -version
java version "1.8.0_66"
Java(TM) SE Runtime Environment (build 1.8.0_66-b17)
Java HotSpot(TM) 64-Bit Server VM (build 25.66-b17, mixed mode)
@rykov
Copy link
Author

rykov commented Feb 5, 2016

I went back and installed MRI 2.2.3 to match jRuby 9.0.5.0 - it also works:

$ rbenv shell 2.2.3 
$ ruby ./example.rb 
OK!

@rykov rykov mentioned this issue Feb 5, 2016
Closed
@enebo enebo added the parser label Feb 5, 2016
@enebo enebo added this to the JRuby 9.1.0.0 milestone Feb 5, 2016
@enebo
Copy link
Member

enebo commented Feb 5, 2016

I am not positive this is a parser issue but something is amiss. Is this also an issue if that file has a bom and some contents?

@rykov
Copy link
Author

rykov commented Feb 6, 2016

Works fine with bom (with or without other content):

$ printf '\xFF\xFE' > empty-file
$ jruby ./example.rb 
OK!
$ printf '\xFF\xFE hello' > empty-file
$ jruby ./example.rb 
OK!

@headius
Copy link
Member

headius commented Feb 13, 2016

Looking into this.

headius added a commit that referenced this issue Feb 13, 2016
@headius
Clear enc2 if BOM strip does not produce anything. Fixes #3647.
9698ca4
@headius headius closed this as completed Feb 13, 2016
@headius
Copy link
Member

headius commented Feb 13, 2016

Minor bit of code missing from our BOM stripping logic, to clear the external encoding if no BOM could be found. Unexcluded a test from CRuby that's now passing. Yay!

@oozzal
Copy link

oozzal commented Feb 16, 2016

👍

@pobocks
Copy link

pobocks commented Feb 23, 2016

I have jruby-9.1.0.0-SNAPSHOT (JRUBY_REVISION=cea55fb) from RVM, failing with this error, without there being a BOM in sight, as far as I can tell.

Encoding::ConverterNotFoundError: code converter not found (UTF-8 to UTF-8)
/home/aspace_proc/aspace_processor/shared/bundle/jruby/2.2.0/gems/dotenv-2.1.0/lib/dotenv.rb:59:in `ignoring_nonexistent_files'
/home/aspace_proc/aspace_processor/shared/bundle/jruby/2.2.0/gems/dotenv-2.1.0/lib/dotenv.rb:14:in `block in load'
/home/aspace_proc/aspace_processor/shared/bundle/jruby/2.2.0/gems/dotenv-2.1.0/lib/dotenv.rb:46:in `block in with'
/home/aspace_proc/aspace_processor/shared/bundle/jruby/2.2.0/gems/dotenv-2.1.0/lib/dotenv.rb:45:in `with'
/home/aspace_proc/aspace_processor/shared/bundle/jruby/2.2.0/gems/dotenv-2.1.0/lib/dotenv.rb:13:in `load'
/home/aspace_proc/aspace_processor/shared/bundle/jruby/2.2.0/gems/dotenv-rails-2.1.0/lib/dotenv/rails.rb:29:in `load'
/home/aspace_proc/aspace_processor/shared/bundle/jruby/2.2.0/gems/dotenv-rails-2.1.0/lib/dotenv/rails.rb:44:in `load'
/home/aspace_proc/aspace_processor/shared/bundle/jruby/2.2.0/gems/dotenv-rails-2.1.0/lib/dotenv/rails.rb:20:in `block in Railtie'
/home/aspace_proc/aspace_processor/shared/bundle/jruby/2.2.0/gems/activesupport-4.2.4/lib/active_support/lazy_load_hooks.rb:36:in `execute_hook'
/home/aspace_proc/aspace_processor/shared/bundle/jruby/2.2.0/gems/activesupport-4.2.4/lib/active_support/lazy_load_hooks.rb:45:in `block in run_load_hooks'
/home/aspace_proc/aspace_processor/shared/bundle/jruby/2.2.0/gems/activesupport-4.2.4/lib/active_support/lazy_load_hooks.rb:44:in `run_load_hooks'
/home/aspace_proc/aspace_processor/shared/bundle/jruby/2.2.0/gems/railties-4.2.4/lib/rails/application.rb:148:in `run_load_hooks!'
/home/aspace_proc/aspace_processor/shared/bundle/jruby/2.2.0/gems/railties-4.2.4/lib/rails/application.rb:96:in `instance'
/home/aspace_proc/aspace_processor/shared/bundle/jruby/2.2.0/gems/railties-4.2.4/lib/rails/railtie.rb:123:in `config'
/home/aspace_proc/aspace_processor/releases/20160223175950/config/application.rb:24:in `<class:Application>'
/home/aspace_proc/aspace_processor/releases/20160223175950/config/application.rb:10:in `<module:AspaceProcessor>'
/home/aspace_proc/aspace_processor/releases/20160223175950/config/application.rb:9:in `<top>'
/home/aspace_proc/aspace_processor/releases/20160223175950/Rakefile:1:in `<top>'
/home/aspace_proc/aspace_processor/releases/20160223175950/Rakefile:4:in `(root)'
(See full trace by running task with --trace)

@rykov
Copy link
Author

rykov commented Feb 23, 2016

@pobocks Can you try running the example code in the original post to confirm it's the same bug

@pobocks
Copy link

pobocks commented Feb 23, 2016

Will do - putting a BOM in my .env allowed the process to continue, so it
seems likely to be the same kind of thing, for sure.
On Feb 23, 2016 1:33 PM, "Michael Rykov" notifications@github.com wrote:

@pobocks https://github.com/pobocks Can you try running the example
code in the original post to confirm it's the same bug

—
Reply to this email directly or view it on GitHub
#3647 (comment).

@pobocks
Copy link

pobocks commented Feb 23, 2016

Confirmed:

$ touch empty-file
$ rvm jruby-head do ruby example.rb 
Encoding::ConverterNotFoundError: code converter not found (UTF-8 to UTF-8)
                 read at org/jruby/RubyIO.java:2972
                 read at org/jruby/RubyIO.java:2950
  block in example.rb at example.rb:2
                 open at org/jruby/RubyIO.java:1124
                <top> at example.rb:2

@pobocks
Copy link

pobocks commented Feb 23, 2016

Replicated on an Amazon linux box (probably whatever their Red Hat one is) and OSX 10.9.5 local.

@bkeepers bkeepers mentioned this issue Feb 26, 2016
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
parser
Projects
None yet
Milestone
JRuby 9.1.0.0
Development

No branches or pull requests
5 participants
@headius @enebo @pobocks @rykov @oozzal