-
-
Notifications
You must be signed in to change notification settings - Fork 925
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ArgumentError: string contains null byte #3907
Comments
Oh, lovely. Can you try to coax Bundler into giving us a full backtrace? And once you do that get a 'full' trace with |
https://github.com/jruby/jruby/blob/master/core/src/main/java/org/jruby/util/StringSupport.java#L718 final RubyString s = ptr.convertToString(); backtrace org.jruby.util.StringSupport.checkEmbeddedNulls(StringSupport.java:720)
org.jruby.RubyFile.fileResource(RubyFile.java:1391)
org.jruby.RubyFileTest.file_p(RubyFileTest.java:134)
org.jruby.RubyFileTest$FileTestFileMethods.file_p(RubyFileTest.java:379)
org.jruby.RubyFileTest$FileTestFileMethods$INVOKER$s$1$0$file_p.call(RubyFileTest$FileTestFileMethods$INVOKER$s$1$0$file_p.gen)
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:161)
org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:313)
org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:77)
org.jruby.ir.interpreter.Interpreter.INTERPRET_BLOCK(Interpreter.java:132)
org.jruby.runtime.MixedModeIRBlockBody.commonYieldPath(MixedModeIRBlockBody.java:148)
org.jruby.runtime.IRBlockBody.doYield(IRBlockBody.java:179)
org.jruby.runtime.BlockBody.yield(BlockBody.java:112)
org.jruby.runtime.Block.yield(Block.java:167)
org.jruby.RubyArray.detectCommon(RubyArray.java:4132)
org.jruby.RubyArray.find(RubyArray.java:4105)
org.jruby.RubyEnumerable.find(RubyEnumerable.java:633)
org.jruby.RubyEnumerable$INVOKER$s$find.call(RubyEnumerable$INVOKER$s$find.gen)
org.jruby.internal.runtime.methods.JavaMethod$JavaMethodZeroOrOneBlock.call(JavaMethod.java:513)
org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:77)
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:83)
org.jruby.ir.instructions.CallBase.interpret(CallBase.java:423)
org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:345)
org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:77)
org.jruby.ir.interpreter.InterpreterEngine.interpret(InterpreterEngine.java:83)
org.jruby.internal.runtime.methods.MixedModeIRMethod.INTERPRET_METHOD(MixedModeIRMethod.java:179)
org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:165)
org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:197)
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:161)
org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:313)
org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:77)
org.jruby.ir.interpreter.InterpreterEngine.interpret(InterpreterEngine.java:77)
org.jruby.internal.runtime.methods.MixedModeIRMethod.INTERPRET_METHOD(MixedModeIRMethod.java:144)
org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:130)
org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:189)
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:129)
org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:329)
org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:77)
org.jruby.internal.runtime.methods.MixedModeIRMethod.INTERPRET_METHOD(MixedModeIRMethod.java:109)
org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:95)
org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:77)
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:83)
org.jruby.ir.instructions.CallBase.interpret(CallBase.java:423)
org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:345)
org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:77)
org.jruby.ir.interpreter.InterpreterEngine.interpret(InterpreterEngine.java:77)
org.jruby.internal.runtime.methods.MixedModeIRMethod.INTERPRET_METHOD(MixedModeIRMethod.java:144)
org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:130)
org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:189)
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:129)
org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:329)
org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:77)
org.jruby.ir.interpreter.InterpreterEngine.interpret(InterpreterEngine.java:77)
org.jruby.internal.runtime.methods.MixedModeIRMethod.INTERPRET_METHOD(MixedModeIRMethod.java:144)
org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:130)
org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:189)
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:129)
org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:329)
org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:77)
org.jruby.ir.interpreter.Interpreter.INTERPRET_BLOCK(Interpreter.java:132)
org.jruby.runtime.MixedModeIRBlockBody.commonYieldPath(MixedModeIRBlockBody.java:148)
org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:66)
org.jruby.runtime.Block.call(Block.java:126)
org.jruby.RubyProc.call(RubyProc.java:342)
org.jruby.RubyProc.call19(RubyProc.java:326)
org.jruby.RubyProc$INVOKER$i$0$0$call19.call(RubyProc$INVOKER$i$0$0$call19.gen)
org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:77)
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:83)
org.jruby.ir.instructions.CallBase.interpret(CallBase.java:423)
org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:345)
org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:77)
org.jruby.internal.runtime.methods.MixedModeIRMethod.INTERPRET_METHOD(MixedModeIRMethod.java:109)
org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:95)
org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:77)
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:83)
org.jruby.ir.instructions.CallBase.interpret(CallBase.java:423)
org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:345)
org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:77)
org.jruby.ir.interpreter.Interpreter.INTERPRET_BLOCK(Interpreter.java:132)
org.jruby.runtime.MixedModeIRBlockBody.commonYieldPath(MixedModeIRBlockBody.java:148)
org.jruby.runtime.IRBlockBody.yieldSpecific(IRBlockBody.java:77)
org.jruby.runtime.Block.yieldSpecific(Block.java:136)
org.jruby.RubyKernel.loop(RubyKernel.java:1290)
org.jruby.RubyKernel$INVOKER$s$0$0$loop.call(RubyKernel$INVOKER$s$0$0$loop.gen)
org.jruby.internal.runtime.methods.JavaMethod$JavaMethodZeroBlock.call(JavaMethod.java:497)
org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:273)
org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:79)
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:83)
org.jruby.ir.instructions.CallBase.interpret(CallBase.java:423)
org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:345)
org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:77)
org.jruby.ir.interpreter.InterpreterEngine.interpret(InterpreterEngine.java:83)
org.jruby.internal.runtime.methods.MixedModeIRMethod.INTERPRET_METHOD(MixedModeIRMethod.java:179)
org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:165)
org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:197)
org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:313)
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:163)
org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:313)
org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:77)
org.jruby.ir.interpreter.Interpreter.INTERPRET_BLOCK(Interpreter.java:132)
org.jruby.runtime.MixedModeIRBlockBody.commonYieldPath(MixedModeIRBlockBody.java:148)
org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:66)
org.jruby.runtime.Block.call(Block.java:126)
org.jruby.RubyProc.call(RubyProc.java:342)
org.jruby.RubyProc.call(RubyProc.java:248)
org.jruby.internal.runtime.RubyRunnable.run(RubyRunnable.java:104)
java.lang.Thread.run(Thread.java:745) |
So can you reproduce with |
jruby irb(main):009:0> Dir['C:/Users/*'].map{|x| [x, File.exists?(x)]}
=> [["C:/Users/All Users", true],
["C:/Users/Default", true],
["C:/Users/Default User", true],
["C:/Users/desktop.ini", true],
["C:/Users/Pavel Rosick\xC3\xBD", false],
["C:/Users/Public", true]]
irb(main):010:0> File.expand_path('~')
=> "C:/Users/Pavel Rosick\xF8"
irb(main):010:0> File.exist?("C:/Users/Pavel Rosick\xFD".force_encoding('windows-1250'))
=> true mri irb(main):006:0> Dir['C:/Users/*'].map{|x| [x, File.exists?(x)]}
=> [["C:/Users/All Users", true],
["C:/Users/Default", true],
["C:/Users/Default User", true],
["C:/Users/desktop.ini", true],
["C:/Users/Pavel Rosick\xEC", true],
["C:/Users/Public", true]]
irb(main):013:0* File.expand_path('~')
=> "C:/Users/Pavel Rosick\xFD"
irb(main):010:0> File.exist?("C:/Users/Pavel Rosick\xFD".force_encoding('windows-1250'))
=> true |
simplified case require 'bundler'
Bundler.which("sudo") it looks like MRI uses a filesystem encoding just for ENV['PATH'] but jruby uses an external encoding for all ENV variables C:\Users\Pavel Rosick\xF8\AppData\Roaming\npm jruby irb(main):004:0> ENV['PATH']
=> "C:\\ProgramData\\Oracle\\Java\\javapath;C:\\Windows\\system32;C:\\Windows;C:\\Windows\\System32\\Wbem;C:\\Windows\\System32\\WindowsPowerShell\\v1.0\\;C:\\Program Files\\Git\\cmd;C:\\Ruby22-x64\\bin;C:\\Program Files\\nodejs\\;C:\\phantomjs;C:\\wkhtmltopdf;c:\\Program Files\\apache-maven-3.3.9\\bin;c:\\Program Files\\apache-ant-1.9.7\\bin;C:\\Program Files\\TortoiseGit\\bin;C:\\Program Files\\TortoiseSVN\\bin;C:\\Users\\Pavel Rosick\xF8\\AppData\\Roaming\\npm"
irb(main):006:0> ENV['PATH'].encoding
=> #<Encoding:IBM852> mri irb(main):020:0> ENV['PATH']
=> "C:\\ProgramData\\Oracle\\Java\\javapath;C:\\Windows\\system32;C:\\Windows;C:\\Windows\\System32\\Wbem;C:\\Windows\\System32\\WindowsPowerShell\\v1.0\\;C:\\Program Files\\Git\\cmd;C:\\Ruby22-x64\\bin;C:\\Program Files\\nodejs\\;C:\\phantomjs;C:\\wkhtmltopdf;c:\\Program Files\\apache-maven-3.3.9\\bin;c:\\Program Files\\apache-ant-1.9.7\\bin;C:\\Program Files\\TortoiseGit\\bin;C:\\Program Files\\TortoiseSVN\\bin;C:\\Users\\Pavel Rosick\xEC\\AppData\\Roaming\\npm"
irb(main):021:0> ENV['PATH'].encoding
=> #<Encoding:Windows-1250> |
Oh, well that should be simple enough to fix if that's all there is to it! |
@ahorek So you're saying it only uses a different encoding for the |
You are apparently right! static VALUE
rb_f_getenv(VALUE obj, VALUE name)
{
const char *nam, *env;
nam = env_name(name);
env = getenv(nam);
if (env) {
if (ENVMATCH(nam, PATH_ENV) && !env_path_tainted(env)) {
return env_path_str_new(env);
}
return env_str_new2(env);
}
return Qnil;
} |
I have a patch for you to try: diff --git a/core/src/main/java/org/jruby/util/OSEnvironment.java b/core/src/main/java/org/jruby/util/OSEnvironment.java
index f167fc6..0b969c5 100644
--- a/core/src/main/java/org/jruby/util/OSEnvironment.java
+++ b/core/src/main/java/org/jruby/util/OSEnvironment.java
@@ -89,15 +89,15 @@ public class OSEnvironment {
private static Map<RubyString, RubyString> asMapOfRubyStrings(final Ruby runtime, final Map<?, ?> map) {
@SuppressWarnings("unchecked")
final Map<RubyString, RubyString> rubyMap = new HashMap(map.size() + 2);
- Encoding encoding = runtime.getEncodingService().getLocaleEncoding();
+ Encoding keyEncoding = runtime.getEncodingService().getLocaleEncoding();
// On Windows, map doesn't have corresponding keys for these
if (Platform.IS_WINDOWS) {
// these may be null when in a restricted environment (JRUBY-6514)
String home = SafePropertyAccessor.getProperty("user.home");
String user = SafePropertyAccessor.getProperty("user.name");
- putRubyKeyValuePair(runtime, rubyMap, "HOME", home == null ? "/" : home, encoding);
- putRubyKeyValuePair(runtime, rubyMap, "USER", user == null ? "" : user, encoding);
+ putRubyKeyValuePair(runtime, rubyMap, "HOME", keyEncoding, home == null ? "/" : home, keyEncoding);
+ putRubyKeyValuePair(runtime, rubyMap, "USER", keyEncoding, user == null ? "" : user, keyEncoding);
}
for (Map.Entry<?, ?> entry : map.entrySet()) {
@@ -111,7 +111,12 @@ public class OSEnvironment {
val = entry.getValue();
if ( ! (val instanceof String) ) continue; // Java devs can stuff non-string objects into env
- putRubyKeyValuePair(runtime, rubyMap, key, (String) val, encoding);
+ Encoding valueEncoding = keyEncoding;
+ if ( key.equals("PATH") ) {
+ valueEncoding = runtime.getEncodingService().getFileSystemEncoding();
+ }
+
+ putRubyKeyValuePair(runtime, rubyMap, key, keyEncoding, (String) val, valueEncoding);
}
return rubyMap;
@@ -119,9 +124,9 @@ public class OSEnvironment {
private static void putRubyKeyValuePair(Ruby runtime,
final Map<RubyString, RubyString> map,
- String key, String value, Encoding encoding) {
- ByteList keyBytes = new ByteList(key.getBytes(), encoding);
- ByteList valueBytes = new ByteList(value.getBytes(), encoding);
+ String key, Encoding keyEncoding, String value, Encoding valueEncoding) {
+ ByteList keyBytes = RubyString.encodeBytelist(key, keyEncoding);
+ ByteList valueBytes = RubyString.encodeBytelist(value, valueEncoding);
RubyString keyString = runtime.newString(keyBytes);
RubyString valueString = runtime.newString(valueBytes); |
@headius even if the encoding seems to be correct now, after requiring bundler it's changed again irb(main):001:0> ENV["PATH"].encoding
=> #<Encoding:Windows-1250>
irb(main):002:0> require 'bundler'
=> true
irb(main):003:0> ENV["PATH"].encoding
=> #<Encoding:IBM852> paths = ENV["PATH"]
paths.split(File::PATH_SEPARATOR).find do |path|
puts path => "C:\Users\Pavel Rosickř\AppData\Roaming\npm" #<Encoding:IBM852>
executable_path = File.expand_path(executable, path)
puts executable_path => "C:/Users/Pavel Rosick" #<Encoding:Windows-1250>
File.file?(executable_path) => error
end
ArgumentError: string contains null byte
from org/jruby/RubyFileTest.java:134:in `file?'
from C:/jruby_repo2/jruby/lib/ruby/gems/shared/gems/bundler-1.12.4/lib/bundler.rb:313:in `block in which'
from org/jruby/RubyEnumerable.java:633:in `find'
from C:/jruby_repo2/jruby/lib/ruby/gems/shared/gems/bundler-1.12.4/lib/bundler.rb:308:in `which'
from (irb):2:in `<eval>'
from org/jruby/RubyKernel.java:983:in `eval'
from org/jruby/RubyKernel.java:1290:in `loop'
from org/jruby/RubyKernel.java:1103:in `catch'
from org/jruby/RubyKernel.java:1103:in `catch'
from jirb:13:in `<top>' |
Ahh, I see that MRI also guards writes to this env var with the same encoding check, and Bundler writes to PATH: def set_path
paths = (ENV["PATH"] || "").split(File::PATH_SEPARATOR)
paths.unshift "#{Bundler.bundle_path}/bin"
ENV["PATH"] = paths.uniq.join(File::PATH_SEPARATOR)
end |
Fixes the rest of jruby#3907.
WRT testing this... At least on JDK I discovered that the "locale" encoding (which we pull out of JNIEXPORT jstring JNICALL
Java_java_io_Console_encoding(JNIEnv *env, jclass cls)
{
char buf[64];
int cp = GetConsoleCP();
if (cp >= 874 && cp <= 950)
sprintf(buf, "ms%d", cp);
else
sprintf(buf, "cp%d", cp);
return JNU_NewStringPlatform(env, buf);
} Which is similar to this code from MRI: int
Init_enc_set_filesystem_encoding(void)
{
int idx;
#if defined NO_LOCALE_CHARMAP
# error NO_LOCALE_CHARMAP defined
#elif defined _WIN32 || defined __CYGWIN__
char cp[SIZEOF_CP_NAME];
CP_FORMAT(cp, AreFileApisANSI() ? GetACP() : GetOEMCP());
idx = rb_enc_find_index(cp);
if (idx < 0) idx = ENCINDEX_ASCII;
#else
idx = rb_enc_to_index(rb_default_external_encoding());
#endif
return idx;
} So the only way to test this will be to test it on Windows, but I don't think ruby/spec runs green on Windows right now :-( |
one more thing, ArgumentError: string contains null byte exception is originaly caused by this line |
👍 |
Fixes the rest of jruby#3907.
One additional tweak was required: MRI only does case-insensitive comparison on Windows. |
Environment
irb(main):002:0> ENV_JAVA['file.encoding']
=> Cp1250
irb(main):003:0> Encoding.find('filesystem')
=> Windows-1250
irb(main):001:0> Encoding.default_internal
=> nil
irb(main):002:0> Encoding.default_external
=> #Encoding:IBM852
irb(main):009:0> File.expand_path('
')').encoding=> "C:/Users/Pavel Rosick\xF8"
irb(main):010:0> File.expand_path('
=> #Encoding:Windows-1250
C:\jruby-9.1.1.0\bin>gem list bundler
*** LOCAL GEMS ***
bundler (1.12.4)
jruby 9.1.1.0 (2.3.0) 2016-05-19 fe84e89 Java HotSpot(TM) 64-Bit Server VM 25.92-b14 on 1.8.0_92-b14 +jit [mswin32-x86_64]
related: #3885, #3849
Expected Behavior
it works as expected with -J-Dfile.encoding=utf-8
Actual Behavior
ArgumentError and blank lines
The text was updated successfully, but these errors were encountered: