UDPSocket.new(Socket::AF_INET6) throws exception (JRuby 9.1.14.0) #4869

jordansissel · 2017-11-27T21:12:52Z

Environment

Provide at least:

JRuby version: 9.1.14.0
Operating system and platform (e.g. uname -a): Linux xyz 4.13.12-300.fc27.x86_64 break script engine #1 SMP Wed Nov 8 16:38:01 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

The example reproduction works correctly in JRuby 9.1.13.0

Expected Behavior

Describe your expectation of how JRuby should behave, perhaps by showing how CRuby/MRI behaves.

Creating a UDPSocket for IPv6 throws a Java exception.

Provide an executable Ruby script or a link to an example repository.

% ruby --version
jruby 9.1.14.0 (2.3.3) 2017-11-08 2176f24 OpenJDK 64-Bit Server VM 25.151-b12 on 1.8.0_151-b12 +jit [linux-x86_64]

% ruby -rsocket -e 'UDPSocket.new(Socket::AF_INET6)'
<crashes w/ exception>

Actual Behavior

Describe or show the actual behavior.

% ruby -rsocket -e 'UDPSocket.new(Socket::AF_INET6)'
Unhandled Java exception: java.lang.UnsupportedOperationException: IPv6 not available
java.lang.UnsupportedOperationException: IPv6 not available
               <init> at sun/nio/ch/DatagramChannelImpl.java:138
  openDatagramChannel at sun/nio/ch/SelectorProviderImpl.java:46
                 open at java/nio/channels/DatagramChannel.java:182
           initialize at org/jruby/ext/socket/RubyUDPSocket.java:126
           initialize at org/jruby/ext/socket/RubyUDPSocket.java:116
                 call at org/jruby/ext/socket/RubyUDPSocket$INVOKER$i$initialize.gen:-1
                 call at org/jruby/internal/runtime/methods/JavaMethod.java:739
         cacheAndCall at org/jruby/runtime/callsite/CachingCallSite.java:278
                 call at org/jruby/runtime/callsite/CachingCallSite.java:79
          newInstance at org/jruby/RubyClass.java:1022
          newInstance at org/jruby/RubyIO.java:875
                 call at org/jruby/RubyIO$INVOKER$s$0$0$newInstance.gen:-1
                 call at org/jruby/internal/runtime/methods/DynamicMethod.java:204
                 call at org/jruby/internal/runtime/methods/DynamicMethod.java:200
         cacheAndCall at org/jruby/runtime/callsite/CachingCallSite.java:318
                 call at org/jruby/runtime/callsite/CachingCallSite.java:155
     invokeOther0:new at -e:1
               <main> at -e:1
  invokeWithArguments at java/lang/invoke/MethodHandle.java:627
                 load at org/jruby/ir/Compiler.java:95
            runScript at org/jruby/Ruby.java:828
          runNormally at org/jruby/Ruby.java:747
          runNormally at org/jruby/Ruby.java:765
          runFromMain at org/jruby/Ruby.java:578
        doRunFromMain at org/jruby/Main.java:417
          internalRun at org/jruby/Main.java:305
                  run at org/jruby/Main.java:232
                 main at org/jruby/Main.java:204

The text was updated successfully, but these errors were encountered:

This reverts commit d38e4ff. (Original commit was part of #8620) The revert is due to `UDPSocket.new(Socket::AF_INET)` crashing in 9.1.14.0: * #8736 * jruby/jruby#4869

This reverts commit d38e4ff. (Original commit was part of #8620) The revert is due to `UDPSocket.new(Socket::AF_INET)` crashing in 9.1.14.0: * #8736 * jruby/jruby#4869 Fixes #8738

headius · 2017-11-27T23:45:42Z

This error appears to be coming from the JDK. Does IPv6 UDP work from Java?

headius · 2017-11-28T00:00:14Z

The JDK lines are pretty straightforward...seems like IPv6 is just not available. But you say it works in 9.1.13, so that's very confusing.

https://github.com/frohoff/jdk8u-jdk/blob/master/src/share/classes/sun/nio/ch/DatagramChannelImpl.java#L137-L139

We should handle that error better, obviously. I'll fix that now.

No theories as to why IPv6 suddenly is unavailable in 9.1.14.0 :-( It really doesn't seem like something we could do.

headius · 2017-11-28T00:32:15Z

Ok, I can reproduce it if I tell the JDK to prefer IPv4. Perhaps you are setting this in some environment, or otherwise unsetting it for 9.1.13?

(This is with the new error)

$ jruby -J-Djava.net.preferIPv4Stack=true -rsocket -e "UDPSocket.new(Socket::AF_INET6)"
Errno::EAFNOSUPPORT: Address family not supported by protocol family - socket(2) - udp
  initialize at org/jruby/ext/socket/RubyUDPSocket.java:134
  initialize at org/jruby/ext/socket/RubyUDPSocket.java:116
         new at org/jruby/RubyIO.java:877
      <main> at -e:1

headius · 2017-11-28T00:38:22Z

FWIW I did not find this property in either our bash script or or native launcher.

It might be getting set in some CI environments to avoid issues with IPv4/IPv6 address resolution (JDK tended to prefer IPv6 for binding while MRI preferred IPv4).

jordansissel · 2017-11-28T06:49:10Z

Nothing I can tell -- I am using rbenv, and from the same shell:

% rbenv shell jruby-9.1.14.0
% ruby --version
jruby 9.1.14.0 (2.3.3) 2017-11-08 2176f24 OpenJDK 64-Bit Server VM 25.151-b12 on 1.8.0_151-b12 +jit [linux-x86_64]

% ruby -rsocket -e 'UDPSocket.new(Socket::AF_INET6)'
Unhandled Java exception: java.lang.UnsupportedOperationException: IPv6 not available
...

% rbenv shell jruby-9.1.13.0
% ruby --version
jruby 9.1.13.0 (2.3.3) 2017-09-06 8e1c115 OpenJDK 64-Bit Server VM 25.151-b12 on 1.8.0_151-b12 +jit [linux-x86_64]
% ruby -rsocket -e 'UDPSocket.new(Socket::AF_INET6)'
<no output, successful exit>

If I throw in a sleep so I can check the java args with jps sleep 10; UDPSocket.new(Socket:AF_INET6), I show the following:

Jruby 9.1.14.0

239720 Main -Djdk.home= -Djruby.home=/home/jls/.rbenv/versions/jruby-9.1.14.0 -Djruby.script=jruby -Djruby.shell=/bin/sh -Djffi.boot.library.path=/home/jls/.rbenv/versions/jruby-9.1.14.0/lib/jni:/home/jls/.rbenv/versions/jruby-9.1.14.0/lib/jni/sparcv9-Linux:/home/jls/.rbenv/versions/jruby-9.1.14.0/lib/jni/ppc64le-Linux:/home/jls/.rbenv/versions/jruby-9.1.14.0/lib/jni/aarch64-Linux:/home/jls/.rbenv/versions/jruby-9.1.14.0/lib/jni/x86_64-Linux:/home/jls/.rbenv/versions/jruby-9.1.14.0/lib/jni/ppc64-Linux:/home/jls/.rbenv/versions/jruby-9.1.14.0/lib/jni/i386-Linux:/home/jls/.rbenv/versions/jruby-9.1.14.0/lib/jni/arm-Linux -Xmx500m -Xss2048k -Xbootclasspath/a:/home/jls/.rbenv/versions/jruby-9.1.14.0/lib/jruby.jar -Djava.security.egd=file:/dev/urandom

JRuby 9.1.13.0

239895 Main -Djdk.home= -Djruby.home=/home/jls/.rbenv/versions/jruby-9.1.13.0 -Djruby.script=jruby -Djruby.shell=/bin/sh -Djffi.boot.library.path=/home/jls/.rbenv/versions/jruby-9.1.13.0/lib/jni:/home/jls/.rbenv/versions/jruby-9.1.13.0/lib/jni/sparcv9-Linux:/home/jls/.rbenv/versions/jruby-9.1.13.0/lib/jni/ppc64le-Linux:/home/jls/.rbenv/versions/jruby-9.1.13.0/lib/jni/aarch64-Linux:/home/jls/.rbenv/versions/jruby-9.1.13.0/lib/jni/x86_64-Linux:/home/jls/.rbenv/versions/jruby-9.1.13.0/lib/jni/ppc64-Linux:/home/jls/.rbenv/versions/jruby-9.1.13.0/lib/jni/i386-Linux:/home/jls/.rbenv/versions/jruby-9.1.13.0/lib/jni/arm-Linux -Xmx500m -Xss2048k -Xbootclasspath/a:/home/jls/.rbenv/versions/jruby-9.1.13.0/lib/jruby.jar

For easier viewing, here's a diff of the arguments (sorted):

--- /tmp/jps.13 2017-11-27 19:59:58.496754680 -0800
+++ /tmp/jps.14 2017-11-27 20:00:01.386075726 -0800
@@ -1,9 +1,10 @@
+-Djava.security.egd=file:/dev/urandom
 -Djdk.home=
--Djffi.boot.library.path=/home/jls/.rbenv/versions/jruby-9.1.13.0/lib/jni:/home/jls/.rbenv/versions/jruby-9.1.13.0/lib/jni/sparcv9-Linux:/home/jls/.rbenv/versions/jruby-9.1.13.0/lib/jni/ppc64le-Linux:/home/jls/.rbenv/versions/jruby-9.1.13.0/lib/jni/aarch64-Linux:/home/jls/.rbenv/versions/jruby-9.1.13.0/lib/jni/x86_64-Linux:/home/jls/.rbenv/versions/jruby-9.1.13.0/lib/jni/ppc64-Linux:/home/jls/.rbenv/versions/jruby-9.1.13.0/lib/jni/i386-Linux:/home/jls/.rbenv/versions/jruby-9.1.13.0/lib/jni/arm-Linux
--Djruby.home=/home/jls/.rbenv/versions/jruby-9.1.13.0
+-Djffi.boot.library.path=/home/jls/.rbenv/versions/jruby-9.1.14.0/lib/jni:/home/jls/.rbenv/versions/jruby-9.1.14.0/lib/jni/sparcv9-Linux:/home/jls/.rbenv/versions/jruby-9.1.14.0/lib/jni/ppc64le-Linux:/home/jls/.rbenv/versions/jruby-9.1.14.0/lib/jni/aarch64-Linux:/home/jls/.rbenv/versions/jruby-9.1.14.0/lib/jni/x86_64-Linux:/home/jls/.rbenv/versions/jruby-9.1.14.0/lib/jni/ppc64-Linux:/home/jls/.rbenv/versions/jruby-9.1.14.0/lib/jni/i386-Linux:/home/jls/.rbenv/versions/jruby-9.1.14.0/lib/jni/arm-Linux
+-Djruby.home=/home/jls/.rbenv/versions/jruby-9.1.14.0
 -Djruby.script=jruby
 -Djruby.shell=/bin/sh
 Main
--Xbootclasspath/a:/home/jls/.rbenv/versions/jruby-9.1.13.0/lib/jruby.jar
+-Xbootclasspath/a:/home/jls/.rbenv/versions/jruby-9.1.14.0/lib/jruby.jar
 -Xmx500m
 -Xss2048k

jordansissel · 2017-11-28T06:58:33Z

# JRuby 9.1.13.0
>> java.net.DatagramSocket.new(8888, java.net.InetAddress.getAllByName("::1").first)
=> #<Java::JavaNet::DatagramSocket:0x6aaceffd>

#JRuby 9.1.14.0
>> java.net.DatagramSocket.new(8888, java.net.InetAddress.getAllByName("::1").first)
Java::JavaNet::SocketException: Protocol family unavailable

Fascinating! Inet6 really seems disabled somehow ... strace shows this for the given java.net.DatagramSocket.new(8888, java.net.InetAddress.getAllByName("::1").first) code:

# 9.1.14.0
[pid 240525] socket(AF_INET, SOCK_DGRAM, IPPROTO_IP) = 17
[pid 240525] setsockopt(17, SOL_SOCKET, SO_BROADCAST, [1], 4) = 0
[pid 240525] setsockopt(17, SOL_IP, IP_MULTICAST_ALL, [0], 4) = 0

# 9.1.13.0
[pid 241108] socket(AF_INET6, SOCK_DGRAM, IPPROTO_IP) = 6
[pid 241108] setsockopt(6, SOL_IPV6, IPV6_V6ONLY, [0], 4) = 0
[pid 241108] setsockopt(6, SOL_SOCKET, SO_BROADCAST, [1], 4) = 0
[pid 241108] setsockopt(6, SOL_IPV6, IPV6_RECVPKTINFO, [0], 4) = 0
[pid 241108] setsockopt(6, SOL_IPV6, IPV6_MULTICAST_HOPS, [1], 4) = 0

jordansissel · 2017-11-28T07:02:58Z

% rbenv shell jruby-9.1.14.0
% ruby  -e 'puts java.lang.System.getProperty("java.net.preferIPv4Stack")'
true
% rbenv shell jruby-9.1.13.0
% ruby  -e 'puts java.lang.System.getProperty("java.net.preferIPv4Stack")'
true

I'm going to close this since it is clear my system has preferIPv4 enabled, but honestly I have no idea how. If I find the reason and if that reason is mysterious, I'll post it here for posterity, but I don't think this is a JRuby bug at this time.

From Java directly, System.out.println(System.getProperty("java.net.preferIPv4Stack")) prints null, not true as it does in my ruby tests... this is wild.

jordansissel · 2017-11-28T07:21:55Z

I tested on a fresh docker image ubuntu and downloaded JRuby 9.1.14.0 to reproduce it

% docker run -it ubuntu bash -li
...
root@14a33df15475:/# apt-get update
...
root@14a33df15475:/# apt-get install openjdk-8-jre-headless
...
root@14a33df15475:/# apt-get install wget
...
root@14a33df15475:/# wget https://repo1.maven.org/maven2/org/jruby/jruby-complete/9.1.14.0/jruby-complete-9.1.14.0.jar
...
root@14a33df15475:/# java -jar jruby-complete-9.1.14.0.jar  -e 'puts java.lang.System.getProperty("java.net.preferIPv4Stack")'
true
root@14a33df15475:/# java -jar jruby-complete-9.1.14.0.jar -e 'java.net.DatagramSocket.new(8888, java.net.InetAddress.getAllByName("::1").first)'
Unhandled Java exception: java.net.SocketException: Protocol family unavailable

headius · 2017-11-28T07:29:36Z

@jordansissel Of course I should have grepped JRuby source as well. I discovered 691bf16 meant to address #775, and this does indeed attempt to set the related Java property to prefer IPv4.

Given this, my current theory is that some library or core change to JRuby in 9.1.14.0 has now allowed this setting to actually take effect when it wasn't before. For example, a newer RubyGems may not need to activate the socket library, allowing our settings to kick in before the JDK's socket subsystem boots. This could be the changes I did for JDK9 to defer some reflection until needed.

headius · 2017-11-28T07:31:37Z

If my theory plays out, then this IPv4-forcing has not been working consistently anyway, and we may as well eliminate the configurable setting.

jordansissel · 2017-11-28T07:33:41Z

Of course I should have grepped JRuby source as well

haha, this was the last place I expected to find the answer, fwiw. I spent a bit of time chasing down rbenv, fedora global java settings, jdk source code, etc. It was a good adventure for me! :)

I can at least confirm that in Java, when preferIPv4Stack=true, UDP on IPV6 fails:

% java U
java.net.preferIPv4Stack: null
Socket: java.net.DatagramSocket@4e25154f

% java -Djava.net.preferIPv4Stack=true U
java.net.preferIPv4Stack: true
Exception in thread "main" java.net.SocketException: Protocol family unavailable

I think your theory makes sense. My gut says that y'all fixed some things in 9.1.14.0 that accidentally enabled this feature that was (for years!) intended to have this effect.

Let me know if I can help further. I'm open to sending a patch also, if you want to spread that load. I am also happy to test any patches if I am not authoring.

headius · 2017-11-28T07:41:53Z

It's possible the feature did work originally, but as time went on it became less of a concern. In any case, I think it may be better to just relegate this setting to JAVA_OPTS and remove it from our properties.

headius · 2017-11-28T07:45:42Z

The related networking properties: https://docs.oracle.com/javase/8/docs/api/java/net/doc-files/net-properties.html

This reverts commit d38e4ff. (Original commit was part of elastic#8620) The revert is due to `UDPSocket.new(Socket::AF_INET)` crashing in 9.1.14.0: * elastic#8736 * jruby/jruby#4869 Fixes elastic#8738

jordansissel mentioned this issue Nov 27, 2017

master branch failing ipv6 elastic/logstash#8736

Closed

jordansissel mentioned this issue Nov 27, 2017

Revert "Upgrade to JRuby 9.1.14.0" elastic/logstash#8738

Closed

headius added a commit that referenced this issue Nov 28, 2017

Better exception for unavailable IPv6. #4869

e43ee25

headius added this to the JRuby 9.1.15.0 milestone Nov 28, 2017

headius added the stdlib label Nov 28, 2017

jordansissel closed this as completed Nov 28, 2017

jordansissel reopened this Nov 28, 2017

headius closed this as completed in d55cd48 Nov 29, 2017

original-brownbear mentioned this issue Jan 29, 2018

[WIP]Upgrade to JRuby 9.1.15.0 elastic/logstash#9053

Closed

jeremyevans mentioned this issue Feb 21, 2018

java.net.preferIPv4Stack=true warning not possible to disable on OpenBSD #5057

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

GitHub Sponsors

UDPSocket.new(Socket::AF_INET6) throws exception (JRuby 9.1.14.0) #4869

UDPSocket.new(Socket::AF_INET6) throws exception (JRuby 9.1.14.0) #4869

jordansissel commented Nov 27, 2017 •

edited

Loading

headius commented Nov 27, 2017

headius commented Nov 28, 2017

headius commented Nov 28, 2017

headius commented Nov 28, 2017

jordansissel commented Nov 28, 2017 •

edited

Loading

jordansissel commented Nov 28, 2017

jordansissel commented Nov 28, 2017 •

edited

Loading

jordansissel commented Nov 28, 2017

headius commented Nov 28, 2017

headius commented Nov 28, 2017

jordansissel commented Nov 28, 2017

headius commented Nov 28, 2017

headius commented Nov 28, 2017

UDPSocket.new(Socket::AF_INET6) throws exception (JRuby 9.1.14.0) #4869

UDPSocket.new(Socket::AF_INET6) throws exception (JRuby 9.1.14.0) #4869

Comments

jordansissel commented Nov 27, 2017 • edited Loading

Environment

Expected Behavior

Actual Behavior

headius commented Nov 27, 2017

headius commented Nov 28, 2017

headius commented Nov 28, 2017

headius commented Nov 28, 2017

jordansissel commented Nov 28, 2017 • edited Loading

jordansissel commented Nov 28, 2017

jordansissel commented Nov 28, 2017 • edited Loading

jordansissel commented Nov 28, 2017

headius commented Nov 28, 2017

headius commented Nov 28, 2017

jordansissel commented Nov 28, 2017

headius commented Nov 28, 2017

headius commented Nov 28, 2017

jordansissel commented Nov 27, 2017 •

edited

Loading

jordansissel commented Nov 28, 2017 •

edited

Loading

jordansissel commented Nov 28, 2017 •

edited

Loading