@mmustala Thanks for this.

@mmustala I am wondering what mode you are running this in and whether perhaps your repro is incomplete. If I reduce your snippet to:

filename = "neliö"

I reduced it to this because the parser never even makes it to the backtick line. So, I see the error you are mentioning without backticks + interpolation in the script. However, in this case we are correct since MRI 1.9 also raises the same error:

% ./bin/jruby --1.8 snippets/enc1.rb
% ./bin/jruby --1.9 snippets/enc1.rb
SyntaxError: snippets/enc1.rb:2: invalid multibyte char (US-ASCII)
% ./bin/jruby --2.0 snippets/enc1.rb

vs mri:

% mri18 snippets/enc1.rb
% mri19 snippets/enc1.rb
snippets/enc1.rb:1: invalid multibyte char (US-ASCII)
snippets/enc1.rb:1: invalid multibyte char (US-ASCII)
% mri20 snippets/enc1.rb

Mysteriously 1.9.3 seems to output the syntax error twice but we basically behave the same.

So I definitely believe we probably have a bug with backticks, interpolation, and encoding but your repro is not it. Could you take another swipe at a repro? I would love to fix any missing encoding issue we have. A panda dies everytime I see someone submits a workaround fix for JRuby :)

Sorry, I had just taken those lines from my irb. In a file it requires the encoding comment on the first line.

#encoding: utf-8
filename = "neliö" # Any string with multibyte chars should do
File.write(filename, "content")
mime = `file -b --mime #{filename}`
puts mime
# => "ERROR: cannot open `neliö' (No such file or directory)\n"

This should run and reproduce the issue. I'm running on default --1.9 mode and JRuby 1.7.16.1.

AHA! This error message is the result of executing the command with the wrong encoding interpolation. I kept viewing this as an error while interpolating to create the backtick string. Now this makes a lot more sense.

Simple our dxstr was not supplying the lexers encoding when constructing the AST node. @mmustala thanks for providing this. We had really thought we were down to a few esoteric encoding problems and not something this massive...

I remember when I noticed this I thought this cannot be true, JRuby cannot have this bug. Then I read the source where backticks is handled and it said something like "It is on users responsibility to use correct encoding". I thought that I just need to change the encoding in my code then.

But I'm happy to be able to delete those encoding parts from my code soon.