Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ArgumentError: string contains null byte #3907

Closed
ahorek opened this issue May 20, 2016 · 18 comments
Closed

ArgumentError: string contains null byte #3907

ahorek opened this issue May 20, 2016 · 18 comments

Comments

@ahorek
Copy link
Contributor

ahorek commented May 20, 2016

Environment

irb(main):002:0> ENV_JAVA['file.encoding']
=> Cp1250
irb(main):003:0> Encoding.find('filesystem')
=> Windows-1250
irb(main):001:0> Encoding.default_internal
=> nil
irb(main):002:0> Encoding.default_external
=> #Encoding:IBM852
irb(main):009:0> File.expand_path('')
=> "C:/Users/Pavel Rosick\xF8"
irb(main):010:0> File.expand_path('
').encoding
=> #Encoding:Windows-1250
C:\jruby-9.1.1.0\bin>gem list bundler

*** LOCAL GEMS ***

bundler (1.12.4)

jruby 9.1.1.0 (2.3.0) 2016-05-19 fe84e89 Java HotSpot(TM) 64-Bit Server VM 25.92-b14 on 1.8.0_92-b14 +jit [mswin32-x86_64]

related: #3885, #3849

Expected Behavior

it works as expected with -J-Dfile.encoding=utf-8

C:\jruby-9.1.1.0\bin\devel>..\jruby -J-Dfile.encoding=utf-8 -S bundle install
Warning: this Gemfile contains multiple primary sources. Using `source` more than once without a block is a security risk, and may result in installing unexpected gems. To resolve this warning, use a block to indicate which gems should come from the secondary source. To upgrade this warning to an error, run `bundle config disable_multisource true`.
Fetching gem metadata from http://rubygems.org/...........
Fetching gem metadata from https://rubygems.org/...........
Fetching version metadata from http://rubygems.org/...
Fetching version metadata from https://rubygems.org/...
Fetching dependency metadata from http://rubygems.org/..
Fetching dependency metadata from https://rubygems.org/..
Resolving dependencies.............................................................
Installing rake 11.1.2
Installing i18n 0.7.0
Using json 1.8.3
Installing minitest 5.9.0
Installing thread_safe 0.3.5

Actual Behavior

ArgumentError and blank lines

C:\jruby-9.1.1.0\bin\devel>..\jruby -S bundle install
Warning: this Gemfile contains multiple primary sources. Using `source` more than once without a block is a security risk, and may result in installing unexpected gems. To resolve this warning, use a block to indicate which gems should come from the secondary source. To upgrade this warning to an error, run `bundle config disable_multisource true`.
Fetching gem metadata from http://rubygems.org/...........
Fetching gem metadata from https://rubygems.org/...........
Fetching version metadata from http://rubygems.org/...
Fetching version metadata from https://rubygems.org/...
Fetching dependency metadata from http://rubygems.org/..
Fetching dependency metadata from https://rubygems.org/..
Resolving dependencies.............................................................

ArgumentError: string contains null byte

Using json 1.8.3











Using bundler 1.12.4





















An error occurred while installing rake (11.1.2), and Bundler cannot continue.
Make sure that `gem install rake -v '11.1.2'` succeeds before bundling.
@headius
Copy link
Member

headius commented May 20, 2016

Oh, lovely. Can you try to coax Bundler into giving us a full backtrace? And once you do that get a 'full' trace with -Xbacktrace.style=full please?

@ahorek
Copy link
Contributor Author

ahorek commented May 20, 2016

https://github.com/jruby/jruby/blob/master/core/src/main/java/org/jruby/util/StringSupport.java#L718
original path: C:/Users/Pavel Rosický

final RubyString s = ptr.convertToString();
s => C:/Users/Pavel Rosick

backtrace

org.jruby.util.StringSupport.checkEmbeddedNulls(StringSupport.java:720)
org.jruby.RubyFile.fileResource(RubyFile.java:1391)
org.jruby.RubyFileTest.file_p(RubyFileTest.java:134)
org.jruby.RubyFileTest$FileTestFileMethods.file_p(RubyFileTest.java:379)
org.jruby.RubyFileTest$FileTestFileMethods$INVOKER$s$1$0$file_p.call(RubyFileTest$FileTestFileMethods$INVOKER$s$1$0$file_p.gen)
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:161)
org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:313)
org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:77)
org.jruby.ir.interpreter.Interpreter.INTERPRET_BLOCK(Interpreter.java:132)
org.jruby.runtime.MixedModeIRBlockBody.commonYieldPath(MixedModeIRBlockBody.java:148)
org.jruby.runtime.IRBlockBody.doYield(IRBlockBody.java:179)
org.jruby.runtime.BlockBody.yield(BlockBody.java:112)
org.jruby.runtime.Block.yield(Block.java:167)
org.jruby.RubyArray.detectCommon(RubyArray.java:4132)
org.jruby.RubyArray.find(RubyArray.java:4105)
org.jruby.RubyEnumerable.find(RubyEnumerable.java:633)
org.jruby.RubyEnumerable$INVOKER$s$find.call(RubyEnumerable$INVOKER$s$find.gen)
org.jruby.internal.runtime.methods.JavaMethod$JavaMethodZeroOrOneBlock.call(JavaMethod.java:513)
org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:77)
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:83)
org.jruby.ir.instructions.CallBase.interpret(CallBase.java:423)
org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:345)
org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:77)
org.jruby.ir.interpreter.InterpreterEngine.interpret(InterpreterEngine.java:83)
org.jruby.internal.runtime.methods.MixedModeIRMethod.INTERPRET_METHOD(MixedModeIRMethod.java:179)
org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:165)
org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:197)
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:161)
org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:313)
org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:77)
org.jruby.ir.interpreter.InterpreterEngine.interpret(InterpreterEngine.java:77)
org.jruby.internal.runtime.methods.MixedModeIRMethod.INTERPRET_METHOD(MixedModeIRMethod.java:144)
org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:130)
org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:189)
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:129)
org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:329)
org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:77)
org.jruby.internal.runtime.methods.MixedModeIRMethod.INTERPRET_METHOD(MixedModeIRMethod.java:109)
org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:95)
org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:77)
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:83)
org.jruby.ir.instructions.CallBase.interpret(CallBase.java:423)
org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:345)
org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:77)
org.jruby.ir.interpreter.InterpreterEngine.interpret(InterpreterEngine.java:77)
org.jruby.internal.runtime.methods.MixedModeIRMethod.INTERPRET_METHOD(MixedModeIRMethod.java:144)
org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:130)
org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:189)
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:129)
org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:329)
org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:77)
org.jruby.ir.interpreter.InterpreterEngine.interpret(InterpreterEngine.java:77)
org.jruby.internal.runtime.methods.MixedModeIRMethod.INTERPRET_METHOD(MixedModeIRMethod.java:144)
org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:130)
org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:189)
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:129)
org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:329)
org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:77)
org.jruby.ir.interpreter.Interpreter.INTERPRET_BLOCK(Interpreter.java:132)
org.jruby.runtime.MixedModeIRBlockBody.commonYieldPath(MixedModeIRBlockBody.java:148)
org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:66)
org.jruby.runtime.Block.call(Block.java:126)
org.jruby.RubyProc.call(RubyProc.java:342)
org.jruby.RubyProc.call19(RubyProc.java:326)
org.jruby.RubyProc$INVOKER$i$0$0$call19.call(RubyProc$INVOKER$i$0$0$call19.gen)
org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:77)
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:83)
org.jruby.ir.instructions.CallBase.interpret(CallBase.java:423)
org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:345)
org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:77)
org.jruby.internal.runtime.methods.MixedModeIRMethod.INTERPRET_METHOD(MixedModeIRMethod.java:109)
org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:95)
org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:77)
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:83)
org.jruby.ir.instructions.CallBase.interpret(CallBase.java:423)
org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:345)
org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:77)
org.jruby.ir.interpreter.Interpreter.INTERPRET_BLOCK(Interpreter.java:132)
org.jruby.runtime.MixedModeIRBlockBody.commonYieldPath(MixedModeIRBlockBody.java:148)
org.jruby.runtime.IRBlockBody.yieldSpecific(IRBlockBody.java:77)
org.jruby.runtime.Block.yieldSpecific(Block.java:136)
org.jruby.RubyKernel.loop(RubyKernel.java:1290)
org.jruby.RubyKernel$INVOKER$s$0$0$loop.call(RubyKernel$INVOKER$s$0$0$loop.gen)
org.jruby.internal.runtime.methods.JavaMethod$JavaMethodZeroBlock.call(JavaMethod.java:497)
org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:273)
org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:79)
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:83)
org.jruby.ir.instructions.CallBase.interpret(CallBase.java:423)
org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:345)
org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:77)
org.jruby.ir.interpreter.InterpreterEngine.interpret(InterpreterEngine.java:83)
org.jruby.internal.runtime.methods.MixedModeIRMethod.INTERPRET_METHOD(MixedModeIRMethod.java:179)
org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:165)
org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:197)
org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:313)
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:163)
org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:313)
org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:77)
org.jruby.ir.interpreter.Interpreter.INTERPRET_BLOCK(Interpreter.java:132)
org.jruby.runtime.MixedModeIRBlockBody.commonYieldPath(MixedModeIRBlockBody.java:148)
org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:66)
org.jruby.runtime.Block.call(Block.java:126)
org.jruby.RubyProc.call(RubyProc.java:342)
org.jruby.RubyProc.call(RubyProc.java:248)
org.jruby.internal.runtime.RubyRunnable.run(RubyRunnable.java:104)
java.lang.Thread.run(Thread.java:745)

@headius
Copy link
Member

headius commented May 20, 2016

So can you reproduce with File.file?("C:/Users/Pavel Rosický")?

@headius headius added this to the JRuby 9.1.2.0 milestone May 20, 2016
@ahorek
Copy link
Contributor Author

ahorek commented May 21, 2016

jruby

irb(main):009:0> Dir['C:/Users/*'].map{|x| [x, File.exists?(x)]}
=> [["C:/Users/All Users", true],
 ["C:/Users/Default", true],
 ["C:/Users/Default User", true],
 ["C:/Users/desktop.ini", true],
 ["C:/Users/Pavel Rosick\xC3\xBD", false],
 ["C:/Users/Public", true]]

irb(main):010:0> File.expand_path('~')
=> "C:/Users/Pavel Rosick\xF8"

irb(main):010:0> File.exist?("C:/Users/Pavel Rosick\xFD".force_encoding('windows-1250'))
=> true

mri

irb(main):006:0> Dir['C:/Users/*'].map{|x| [x, File.exists?(x)]}
=> [["C:/Users/All Users", true],
 ["C:/Users/Default", true],
 ["C:/Users/Default User", true],
 ["C:/Users/desktop.ini", true],
 ["C:/Users/Pavel Rosick\xEC", true],
 ["C:/Users/Public", true]]

irb(main):013:0* File.expand_path('~')
=> "C:/Users/Pavel Rosick\xFD"

irb(main):010:0> File.exist?("C:/Users/Pavel Rosick\xFD".force_encoding('windows-1250'))
=> true

@ahorek
Copy link
Contributor Author

ahorek commented May 21, 2016

simplified case

require 'bundler'
Bundler.which("sudo")

it looks like MRI uses a filesystem encoding just for ENV['PATH'] but jruby uses an external encoding for all ENV variables

C:\Users\Pavel Rosick\xF8\AppData\Roaming\npm

jruby

irb(main):004:0> ENV['PATH']
=> "C:\\ProgramData\\Oracle\\Java\\javapath;C:\\Windows\\system32;C:\\Windows;C:\\Windows\\System32\\Wbem;C:\\Windows\\System32\\WindowsPowerShell\\v1.0\\;C:\\Program Files\\Git\\cmd;C:\\Ruby22-x64\\bin;C:\\Program Files\\nodejs\\;C:\\phantomjs;C:\\wkhtmltopdf;c:\\Program Files\\apache-maven-3.3.9\\bin;c:\\Program Files\\apache-ant-1.9.7\\bin;C:\\Program Files\\TortoiseGit\\bin;C:\\Program Files\\TortoiseSVN\\bin;C:\\Users\\Pavel Rosick\xF8\\AppData\\Roaming\\npm"
irb(main):006:0> ENV['PATH'].encoding
=> #<Encoding:IBM852>

mri

irb(main):020:0> ENV['PATH']
=> "C:\\ProgramData\\Oracle\\Java\\javapath;C:\\Windows\\system32;C:\\Windows;C:\\Windows\\System32\\Wbem;C:\\Windows\\System32\\WindowsPowerShell\\v1.0\\;C:\\Program Files\\Git\\cmd;C:\\Ruby22-x64\\bin;C:\\Program Files\\nodejs\\;C:\\phantomjs;C:\\wkhtmltopdf;c:\\Program Files\\apache-maven-3.3.9\\bin;c:\\Program Files\\apache-ant-1.9.7\\bin;C:\\Program Files\\TortoiseGit\\bin;C:\\Program Files\\TortoiseSVN\\bin;C:\\Users\\Pavel Rosick\xEC\\AppData\\Roaming\\npm"
irb(main):021:0> ENV['PATH'].encoding
=> #<Encoding:Windows-1250>

@headius
Copy link
Member

headius commented May 25, 2016

Oh, well that should be simple enough to fix if that's all there is to it!

@headius
Copy link
Member

headius commented May 25, 2016

@ahorek So you're saying it only uses a different encoding for the PATH env var? Really?

@headius
Copy link
Member

headius commented May 25, 2016

You are apparently right!

static VALUE
rb_f_getenv(VALUE obj, VALUE name)
{
    const char *nam, *env;

    nam = env_name(name);
    env = getenv(nam);
    if (env) {
    if (ENVMATCH(nam, PATH_ENV) && !env_path_tainted(env)) {
        return env_path_str_new(env);
    }
    return env_str_new2(env);
    }
    return Qnil;
}

@headius
Copy link
Member

headius commented May 25, 2016

I have a patch for you to try:

diff --git a/core/src/main/java/org/jruby/util/OSEnvironment.java b/core/src/main/java/org/jruby/util/OSEnvironment.java
index f167fc6..0b969c5 100644
--- a/core/src/main/java/org/jruby/util/OSEnvironment.java
+++ b/core/src/main/java/org/jruby/util/OSEnvironment.java
@@ -89,15 +89,15 @@ public class OSEnvironment {
     private static Map<RubyString, RubyString> asMapOfRubyStrings(final Ruby runtime, final Map<?, ?> map) {
         @SuppressWarnings("unchecked")
         final Map<RubyString, RubyString> rubyMap = new HashMap(map.size() + 2);
-        Encoding encoding = runtime.getEncodingService().getLocaleEncoding();
+        Encoding keyEncoding = runtime.getEncodingService().getLocaleEncoding();

         // On Windows, map doesn't have corresponding keys for these
         if (Platform.IS_WINDOWS) {
             // these may be null when in a restricted environment (JRUBY-6514)
             String home = SafePropertyAccessor.getProperty("user.home");
             String user = SafePropertyAccessor.getProperty("user.name");
-            putRubyKeyValuePair(runtime, rubyMap, "HOME", home == null ? "/" : home, encoding);
-            putRubyKeyValuePair(runtime, rubyMap, "USER", user == null ? "" : user, encoding);
+            putRubyKeyValuePair(runtime, rubyMap, "HOME", keyEncoding, home == null ? "/" : home, keyEncoding);
+            putRubyKeyValuePair(runtime, rubyMap, "USER", keyEncoding, user == null ? "" : user, keyEncoding);
         }

         for (Map.Entry<?, ?> entry : map.entrySet()) {
@@ -111,7 +111,12 @@ public class OSEnvironment {
             val = entry.getValue();
             if ( ! (val instanceof String) ) continue; // Java devs can stuff non-string objects into env

-            putRubyKeyValuePair(runtime, rubyMap, key, (String) val, encoding);
+            Encoding valueEncoding = keyEncoding;
+            if ( key.equals("PATH") ) {
+                valueEncoding = runtime.getEncodingService().getFileSystemEncoding();
+            }
+
+            putRubyKeyValuePair(runtime, rubyMap, key, keyEncoding, (String) val, valueEncoding);
         }

         return rubyMap;
@@ -119,9 +124,9 @@ public class OSEnvironment {

     private static void putRubyKeyValuePair(Ruby runtime,
         final Map<RubyString, RubyString> map,
-        String key, String value, Encoding encoding) {
-        ByteList keyBytes = new ByteList(key.getBytes(), encoding);
-        ByteList valueBytes = new ByteList(value.getBytes(), encoding);
+        String key, Encoding keyEncoding, String value, Encoding valueEncoding) {
+        ByteList keyBytes = RubyString.encodeBytelist(key, keyEncoding);
+        ByteList valueBytes = RubyString.encodeBytelist(value, valueEncoding);

         RubyString keyString = runtime.newString(keyBytes);
         RubyString valueString = runtime.newString(valueBytes);

@headius
Copy link
Member

headius commented May 25, 2016

@eregon @nobu Suggestions for how to spec this? It would require a system where the filesystem encoding is different than the locale encoding, and I'm not sure how to do that from e.g. env vars.

@ahorek
Copy link
Contributor Author

ahorek commented May 25, 2016

@headius
PATH should ignore cases viz https://github.com/jruby/jruby/pull/3923/files#r64532353

even if the encoding seems to be correct now, after requiring bundler it's changed again

irb(main):001:0> ENV["PATH"].encoding
=> #<Encoding:Windows-1250>
irb(main):002:0> require 'bundler'
=> true
irb(main):003:0> ENV["PATH"].encoding
=> #<Encoding:IBM852>
paths = ENV["PATH"]
paths.split(File::PATH_SEPARATOR).find do |path|
  puts path => "C:\Users\Pavel Rosickř\AppData\Roaming\npm" #<Encoding:IBM852>
  executable_path = File.expand_path(executable, path)
  puts executable_path => "C:/Users/Pavel Rosick" #<Encoding:Windows-1250>
  File.file?(executable_path) => error
end

ArgumentError: string contains null byte
        from org/jruby/RubyFileTest.java:134:in `file?'
        from C:/jruby_repo2/jruby/lib/ruby/gems/shared/gems/bundler-1.12.4/lib/bundler.rb:313:in `block in which'
        from org/jruby/RubyEnumerable.java:633:in `find'
        from C:/jruby_repo2/jruby/lib/ruby/gems/shared/gems/bundler-1.12.4/lib/bundler.rb:308:in `which'
        from (irb):2:in `<eval>'
        from org/jruby/RubyKernel.java:983:in `eval'
        from org/jruby/RubyKernel.java:1290:in `loop'
        from org/jruby/RubyKernel.java:1103:in `catch'
        from org/jruby/RubyKernel.java:1103:in `catch'
        from jirb:13:in `<top>'

@headius
Copy link
Member

headius commented May 25, 2016

Ahh, I see that MRI also guards writes to this env var with the same encoding check, and Bundler writes to PATH:

    def set_path
      paths = (ENV["PATH"] || "").split(File::PATH_SEPARATOR)
      paths.unshift "#{Bundler.bundle_path}/bin"
      ENV["PATH"] = paths.uniq.join(File::PATH_SEPARATOR)
    end

headius added a commit to headius/jruby that referenced this issue May 25, 2016
@headius
Copy link
Member

headius commented May 25, 2016

@ahorek I have pushed additional fixes to #3923 that should ensure PATH stays filesystem-encoded. Can you give that branch a try and report back?

@headius
Copy link
Member

headius commented May 25, 2016

WRT testing this...

At least on JDK I discovered that the "locale" encoding (which we pull out of java.lang.Console) and the "filesystem" encoding (Charset.defaultCharset()) are only going to differ on Windows due to differing implementations of the private native method Console.encoding. It is null on "unix", which means it uses Charset.defaultCharset(), so "locale" and "filesystem" encodings come out the same no matter what. On Windows, it uses this:

JNIEXPORT jstring JNICALL
Java_java_io_Console_encoding(JNIEnv *env, jclass cls)
{
    char buf[64];
    int cp = GetConsoleCP();
    if (cp >= 874 && cp <= 950)
        sprintf(buf, "ms%d", cp);
    else
        sprintf(buf, "cp%d", cp);
    return JNU_NewStringPlatform(env, buf);
}

Which is similar to this code from MRI:

int
Init_enc_set_filesystem_encoding(void)
{
    int idx;
#if defined NO_LOCALE_CHARMAP
# error NO_LOCALE_CHARMAP defined
#elif defined _WIN32 || defined __CYGWIN__
    char cp[SIZEOF_CP_NAME];
    CP_FORMAT(cp, AreFileApisANSI() ? GetACP() : GetOEMCP());
    idx = rb_enc_find_index(cp);
    if (idx < 0) idx = ENCINDEX_ASCII;
#else
    idx = rb_enc_to_index(rb_default_external_encoding());
#endif
    return idx;
}

So the only way to test this will be to test it on Windows, but I don't think ruby/spec runs green on Windows right now :-(

@ahorek
Copy link
Contributor Author

ahorek commented May 25, 2016

one more thing, ArgumentError: string contains null byte exception is originaly caused by this line
https://github.com/jruby/jruby/blob/master/core/src/main/java/org/jruby/RubyFile.java#L1643
because .getUnicodeValue() expects UTF-8 input and that's why it breaks the original FS encoding.
If I change .getUnicodeValue() to .toString() + your patches then I'm able to run bundler and install all gems.

@headius
Copy link
Member

headius commented May 25, 2016

@ahorek Ok that makes sense too. I pushed the additional fix for #3923. Once everything looks good, we can merge.

@ahorek
Copy link
Contributor Author

ahorek commented May 25, 2016

👍

headius added a commit to headius/jruby that referenced this issue May 25, 2016
headius added a commit to headius/jruby that referenced this issue May 25, 2016
headius added a commit that referenced this issue May 25, 2016
headius added a commit that referenced this issue May 25, 2016
headius added a commit that referenced this issue May 25, 2016
headius added a commit that referenced this issue May 25, 2016
@headius
Copy link
Member

headius commented May 25, 2016

One additional tweak was required: MRI only does case-insensitive comparison on Windows.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants