Never-ending getaddress() call when using compressed IPv6 nameservers in /etc/resolv.conf #3663

nbarrientos · 2016-02-13T12:07:41Z

Hi,

I managed to reproduce the following bug in a CentOS 7.2 box with only IPv6 nameservers and dual stack, jRuby 1.7.20.1 and java-1.7.0-openjdk-1.7.0.95-2.6.4.0.el7_2. The same code works fine on MRI 2.0.0. Perhaps this has already been fixed in newer versions, as to me the bug looks too obvious to be still alive. Anyway, here we go:

# cat /etc/resolv.conf
# generated by /usr/sbin/dhclient-script
search cern.ch.
nameserver 2001:1458:201:1000::5
nameserver 2001:1458:201:1100::5

The following program and the resolv.conf above makes the interpreter hang for a long time and return a failure:

require 'resolv'
puts Resolv.getaddress 'web.cern.ch'

# time java -cp /usr/share/puppetserver/puppet-server-release.jar clojure.main -m puppetlabs.puppetserver.cli.ruby --config /etc/puppetserver/conf.d -- -e "require 'resolv'; puts Resolv.getaddress 'web.cern.ch'"
Resolv::ResolvError: no address for web.cern.ch
  getaddress at /usr/share/puppetserver/puppet-server-release.jar!/META-INF/jruby.home/lib/ruby/1.9/resolv.rb:98
  getaddress at /usr/share/puppetserver/puppet-server-release.jar!/META-INF/jruby.home/lib/ruby/1.9/resolv.rb:48
      (root) at -e:1
      invoke at jruby_puppet_core.clj:232
      invoke at jruby_puppet_core.clj:226
      invoke at subcommand.clj:38
    doInvoke at ruby.clj:7
      invoke at core.clj:624
      invoke at main.clj:315
    doInvoke at main.clj:420

real    2m51.910s
user    0m21.020s
sys 0m0.654s

These are the last lines in a debugging session before the first exception is raised. The program blocks in the select() call until it times out and ResolvTimeout is raised.

#0:/usr/share/puppetserver/puppet-server-release.jar!/META-INF/jruby.home/lib/ruby/1.9/resolv.rb:676:Resolv::DNS::Requester:-:           if s = @senders[[from,msg.id]]
#0:/usr/share/puppetserver/puppet-server-release.jar!/META-INF/jruby.home/lib/ruby/1.9/resolv.rb:654:Resolv::DNS::Requester:-:           now = Time.now
#0:/usr/share/puppetserver/puppet-server-release.jar!/META-INF/jruby.home/lib/ruby/1.9/resolv.rb:655:Resolv::DNS::Requester:-:           timeout = timelimit - now
#0:/usr/share/puppetserver/puppet-server-release.jar!/META-INF/jruby.home/lib/ruby/1.9/resolv.rb:656:Resolv::DNS::Requester:-:           if timeout <= 0
#0:/usr/share/puppetserver/puppet-server-release.jar!/META-INF/jruby.home/lib/ruby/1.9/resolv.rb:659:Resolv::DNS::Requester:-:           select_result = IO.select(@socks, nil, nil, timeout)
#0:/usr/share/puppetserver/puppet-server-release.jar!/META-INF/jruby.home/lib/ruby/1.9/resolv.rb:660:Resolv::DNS::Requester:-:           if !select_result
#0:/usr/share/puppetserver/puppet-server-release.jar!/META-INF/jruby.home/lib/ruby/1.9/resolv.rb:661:Resolv::DNS::Requester:-:             raise ResolvTimeout
#0:/usr/share/puppetserver/puppet-server-release.jar!/META-INF/jruby.home/lib/ruby/1.9/resolv.rb:661:Resolv::DNS::Requester:^:             raise ResolvTimeout
/usr/share/puppetserver/puppet-server-release.jar!/META-INF/jruby.home/lib/ruby/1.9/resolv.rb:661: `Resolv::ResolvTimeout' (Resolv::ResolvTimeout)

I think that this is happening because during the previous iteration this statement was evaluated as false because

@senders[[from,msg.id]]

was nil. Why was it nil? Because the search keys didn't match, as 'from' contains the uncompressed flavor of the IPv6 address of the DNS server, whereas 'senders' has the compacted form (presumably coming from /etc/resolv.conf):

(rdb:1) p from
["2001:1458:201:1000:0:0:0:5", 53]
(rdb:1) p msg.id
12941
(rdb:1) p @senders
{[["2001:1458:201:1000::5", 53], 12941]=>#<Resolv::DNS::Requester::UnconnectedUDP::Sender:0x4605f6fa @host="2001:1458:201:1000::5", @msg="2\x8D\x01\x00\x00\x01\x00\x00\x00\x00\x00\x00\x03web\x04cern\x02ch\x00\x00\x01\x00\x01", @data=#<Resolv::DNS::Name: web.cern.ch.>, @sock=#<UDPSocket:fd 28>, @port=53>}

This situation leads to the outer loop not stopping (see "unexpected DNS message ignored"), therefore the program executes IO.select again but there's nothing to read anymore so ResolvTimeout is raised. This exception is probably caught by the caller further up and at some point .request is called again, leading to another loop. This situation repeats many times after a few minutes the call returns a ResolvError all the way up back to the user.

Handcrafting resolv.conf so all addreses are expanded there makes resolv.rb happy:

# generated by /usr/sbin/dhclient-script
search cern.ch.
nameserver 2001:1458:201:1000:0:0:0:5
nameserver 2001:1458:201:1100:0:0:0:5

This way the program quickly exits without hanging:

# time java -cp /usr/share/puppetserver/puppet-server-release.jar clojure.main -m puppetlabs.puppetserver.cli.ruby --config /etc/puppetserver/conf.d -- -e "require 'resolv'; puts Resolv.getaddress 'web.cern.ch'"
188.184.9.235

real    0m10.945s
user    0m19.857s
sys 0m0.553s

Configuring a Resolv::DNS object by hand with a compressed IPv6 address seems to work.

# cat /etc/resolv.conf
# generated by /usr/sbin/dhclient-script
search cern.ch.
nameserver 2001:1458:201:1000::5
nameserver 2001:1458:201:1100::5
....
irb(main):008:0* require 'resolv'
=> true
irb(main):009:0> Resolv::DNS.new(:nameserver => '2001:1458:201:1000::5').getaddress "web.cern.ch"
=> #<Resolv::IPv4 188.184.9.235>
irb(main):010:0> Resolv::DNS.new().getaddress "web.cern.ch"
# Hangs...

The text was updated successfully, but these errors were encountered:

nbarrientos · 2016-02-13T16:10:07Z

I can reproduce the problem using the latest jRuby 1.x:

# cat /etc/resolv.conf
; generated by /usr/sbin/dhclient-script
search cern.ch.
nameserver 2001:1458:201:1000::5
nameserver 2001:1458:201:1100::5
# bin/jruby -v
jruby 1.7.24 (1.9.3p551) 2016-01-20 bd68d85 on OpenJDK 64-Bit Server VM 1.7.0_95-mockbuild_2016_01_21_15_30-b00 +jit [linux-amd64]
# bin/jruby -r debug -e "require 'resolv'; puts Resolv.getaddress 'web.cern.ch'"
Debug.rb
Emacs support available.

/root/jruby-1.7.24/lib/ruby/1.9/debug.rb:910 warning: tracing (e.g. set_trace_func) will not capture all events without --debug flag
/root/jruby-1.7.24/lib/ruby/shared/rubygems/core_ext/kernel_require.rb:39:
(rdb:1) tr on
Trace on.
(rdb:1) c
...
#0:/root/jruby-1.7.24/lib/ruby/1.9/resolv.rb:676:Resolv::DNS::Requester:-:           if s = @senders[[from,msg.id]]
#0:/root/jruby-1.7.24/lib/ruby/1.9/resolv.rb:654:Resolv::DNS::Requester:-:           now = Time.now
#0:/root/jruby-1.7.24/lib/ruby/1.9/resolv.rb:655:Resolv::DNS::Requester:-:           timeout = timelimit - now
#0:/root/jruby-1.7.24/lib/ruby/1.9/resolv.rb:656:Resolv::DNS::Requester:-:           if timeout <= 0
#0:/root/jruby-1.7.24/lib/ruby/1.9/resolv.rb:659:Resolv::DNS::Requester:-:           select_result = IO.select(@socks, nil, nil, timeout)
#0:/root/jruby-1.7.24/lib/ruby/1.9/resolv.rb:660:Resolv::DNS::Requester:-:           if !select_result
#0:/root/jruby-1.7.24/lib/ruby/1.9/resolv.rb:661:Resolv::DNS::Requester:-:             raise ResolvTimeout
#0:/root/jruby-1.7.24/lib/ruby/1.9/resolv.rb:661:Resolv::DNS::Requester:^:             raise ResolvTimeout
/root/jruby-1.7.24/lib/ruby/1.9/resolv.rb:661: `Resolv::ResolvTimeout' (Resolv::ResolvTimeout)
    from -e:1:in `(root)'
/root/jruby-1.7.24/lib/ruby/1.9/resolv.rb:661:            raise ResolvTimeout
(rdb:1) p from
["2001:1458:201:1000:0:0:0:5", 53]
(rdb:1) p msg.id
35327
(rdb:1) p @senders
{[["2001:1458:201:1000::5", 53], 35327]=>#<Resolv::DNS::Requester::UnconnectedUDP::Sender:0x1386c56d @host="2001:1458:201:1000::5", @msg="\x89\xFF\x01\x00\x00\x01\x00\x00\x00\x00\x00\x00\x03web\x04cern\x02ch\x00\x00\x01\x00\x01", @data=#<Resolv::DNS::Name: web.cern.ch.>, @sock=#<UDPSocket:fd 20>, @port=53>}
(rdb:1)

On the other hand:

# cat /etc/resolv.conf
; generated by /usr/sbin/dhclient-script
search cern.ch.
nameserver 2001:1458:201:1000:0:0:0:5
nameserver 2001:1458:201:1100:0:0:0:5
# bin/jruby -r debug -e "require 'resolv'; puts Resolv.getaddress 'web.cern.ch'"
Debug.rb
Emacs support available.

/root/jruby-1.7.24/lib/ruby/1.9/debug.rb:910 warning: tracing (e.g. set_trace_func) will not capture all events without --debug flag
/root/jruby-1.7.24/lib/ruby/shared/rubygems/core_ext/kernel_require.rb:39:
(rdb:1) tr on
Trace on.
(rdb:1) c
...
#0:/root/jruby-1.7.24/lib/ruby/1.9/resolv.rb:626:#<Class:Resolv::DNS>:-:         if h = RequestID[key]
#0:/root/jruby-1.7.24/lib/ruby/1.9/resolv.rb:627:#<Class:Resolv::DNS>:-:           h.delete id
#0:/root/jruby-1.7.24/lib/ruby/1.9/resolv.rb:628:#<Class:Resolv::DNS>:-:           if h.empty?
#0:/root/jruby-1.7.24/lib/ruby/1.9/resolv.rb:629:#<Class:Resolv::DNS>:-:             RequestID.delete key
188.184.9.235
#

The program exits normally.

Same as above but stopping to inspect:

(rdb:1) b /root/jruby-1.7.24/lib/ruby/1.9/resolv.rb:676
Set breakpoint 1 at /root/jruby-1.7.24/lib/ruby/1.9/resolv.rb:676
(rdb:1) c
Breakpoint 1, request at /root/jruby-1.7.24/lib/ruby/1.9/resolv.rb:676
/root/jruby-1.7.24/lib/ruby/1.9/resolv.rb:676:          if s = @senders[[from,msg.id]]
(rdb:1) p from
["2001:1458:201:1000:0:0:0:5", 53]
(rdb:1) p msg.id
25133
(rdb:1) p @senders
{[["2001:1458:201:1000:0:0:0:5", 53], 25133]=>#<Resolv::DNS::Requester::UnconnectedUDP::Sender:0x4dbc377 @host="2001:1458:201:1000:0:0:0:5", @msg="b-\x01\x00\x00\x01\x00\x00\x00\x00\x00\x00\x03web\x04cern\x02ch\x00\x00\x01\x00\x01", @data=#<Resolv::DNS::Name: web.cern.ch.>, @sock=#<UDPSocket:fd 15>, @port=53>}
(rdb:1) c
188.184.9.235
#

nbarrientos · 2016-02-13T16:19:55Z

I see the same behaviour when running 9.0.5.0.

nbarrientos · 2016-02-13T16:50:55Z

And the same debugging process but using MRI (which works with compacted addresses):

# cat /etc/resolv.conf
; generated by /usr/sbin/dhclient-script
search cern.ch.
nameserver 2001:1458:201:1000::5
nameserver 2001:1458:201:1100::5
# ruby -v
ruby 2.0.0p598 (2014-11-13) [x86_64-linux]
# ruby -r debug -e "require 'resolv'; puts Resolv.getaddress 'web.cern.ch'"
Debug.rb
Emacs support available.

/usr/share/rubygems/rubygems/core_ext/kernel_require.rb:57:
(rdb:1) b /usr/share/ruby/resolv.rb:691
Set breakpoint 1 at /usr/share/ruby/resolv.rb:691
(rdb:1) c
Breakpoint 1, request at /usr/share/ruby/resolv.rb:691
/usr/share/ruby/resolv.rb:691:          if s = @senders[[from,msg.id]]
(rdb:1) p from
["2001:1458:201:1000::5", 53]
(rdb:1) p msg.id
50581
(rdb:1) p @senders
{[["2001:1458:201:1000::5", 53], 50581]=>#<Resolv::DNS::Requester::UnconnectedUDP::Sender:0x00000002c996e8 @msg="\xC5\x95\x01\x00\x00\x01\x00\x00\x00\x00\x00\x00\x03web\x04cern\x02ch\x00\x00\x01\x00\x01", @data=#<Resolv::DNS::Name: web.cern.ch.>, @sock=#<UDPSocket:fd 7>, @host="2001:1458:201:1000::5", @port=53>}
(rdb:1) c
188.184.9.235
#

nbarrientos · 2016-02-14T10:56:29Z

Hi again,

(It's getting quite amusing to speak to myself :))

It's also interesting to observe that the problem is only triggered when more than one (compressed) IPv6 nameservers are available and therefore UnconnectedUDP is used. I realized about this after trying to reproduce the problem by creating Resolv::DNS objects with manual configuration aiming to take resolv.conf out of the equation.

So first test case, two IPv6 nameservers with compressed addresses. As shown in previous comments this should trigger the bug and therefore fail.

# bin/jruby -r debug -e "require 'resolv'; puts Resolv::DNS.new(:nameserver => ['2001:1458:201:1000::5', '2001:1458:201:1100::5']).getaddress 'web.cern.ch'"
(rdb:1) c
/root/jruby-1.7.24/lib/ruby/1.9/resolv.rb:661: `Resolv::ResolvTimeout' (Resolv::ResolvTimeout)
        from -e:1:in `(root)'
/root/jruby-1.7.24/lib/ruby/1.9/resolv.rb:661:            raise ResolvTimeout
(rdb:1) p from
["2001:1458:201:1000:0:0:0:5", 53]
(rdb:1) p @senders
{[["2001:1458:201:1000::5", 53], 22726]=>#<Resolv::DNS::Requester::UnconnectedUDP::Sender:0x3dcf9961 @host="2001:1458:201:1000::5", @msg="X\xC6\x01\x00\x00\x01\x00\x00\x00\x00\x00\x00\x03web\x04cern\x02ch\x00\x00\x01\x00\x01", @data=#<Resolv::DNS::Name: web.cern.ch.>, @sock=#<UDPSocket:fd 13>, @port=53>}
(rdb:1) c
/root/jruby-1.7.24/lib/ruby/1.9/resolv.rb:661: `Resolv::ResolvTimeout' (Resolv::ResolvTimeout)
        from -e:1:in `(root)'
/root/jruby-1.7.24/lib/ruby/1.9/resolv.rb:661:            raise ResolvTimeout
(rdb:1) c
/root/jruby-1.7.24/lib/ruby/1.9/resolv.rb:661: `Resolv::ResolvTimeout' (Resolv::ResolvTimeout)
        from -e:1:in `(root)'
/root/jruby-1.7.24/lib/ruby/1.9/resolv.rb:661:            raise ResolvTimeout
.... and on and on ....
(rdb:1) q
Really quit? (y/n) y
# FAIL

Now only one compressed nameserver, note that Resolv::DNS::Requester::ConnectedUDP is used instead:

# bin/jruby -r debug -e "require 'resolv'; puts Resolv::DNS.new(:nameserver => '2001:1458:201:1000::5').getaddress 'web.cern.ch'"Debug.rb
Emacs support available.

/root/jruby-1.7.24/lib/ruby/1.9/debug.rb:910 warning: tracing (e.g. set_trace_func) will not capture all events without --debug flag
/root/jruby-1.7.24/lib/ruby/shared/rubygems/core_ext/kernel_require.rb:39:
(rdb:1) b /root/jruby-1.7.24/lib/ruby/1.9/resolv.rb:676
Set breakpoint 1 at /root/jruby-1.7.24/lib/ruby/1.9/resolv.rb:676
(rdb:1) c
Breakpoint 1, request at /root/jruby-1.7.24/lib/ruby/1.9/resolv.rb:676
/root/jruby-1.7.24/lib/ruby/1.9/resolv.rb:676:          if s = @senders[[from,msg.id]]
(rdb:1) p @senders
{[nil, 39392]=>#<Resolv::DNS::Requester::ConnectedUDP::Sender:0x36a70903 @msg="\x99\xE0\x01\x00\x00\x01\x00\x00\x00\x00\x00\x00\x03web\x04cern\x02ch\x00\x00\x01\x00\x01", @sock=#<UDPSocket:fd 13>, @data=#<Resolv::DNS::Name: web.cern.ch.>>}
(rdb:1) p from
nil
(rdb:1) c
188.184.9.235
# SUCCESS

It works!

Finally, the last test case for completeness' sake: more than one IPv6 nameserver without compressed addresses. This should also work as seen before (when I expanded the addresses in resolv.conf by hand).

# bin/jruby -r debug -e "require 'resolv'; puts Resolv::DNS.new(:nameserver => ['2001:1458:201:1000:0:0:0:5', '2001:1458:201:1100:0:0:0:5']).getaddress 'web.cern.ch'"
Debug.rb
Emacs support available.

/root/jruby-1.7.24/lib/ruby/1.9/debug.rb:910 warning: tracing (e.g. set_trace_func) will not capture all events without --debug flag
/root/jruby-1.7.24/lib/ruby/shared/rubygems/core_ext/kernel_require.rb:39:
(rdb:1)  b /root/jruby-1.7.24/lib/ruby/1.9/resolv.rb:676
Set breakpoint 1 at /root/jruby-1.7.24/lib/ruby/1.9/resolv.rb:676
(rdb:1) c
Breakpoint 1, request at /root/jruby-1.7.24/lib/ruby/1.9/resolv.rb:676
/root/jruby-1.7.24/lib/ruby/1.9/resolv.rb:676:          if s = @senders[[from,msg.id]]
(rdb:1) p from
["2001:1458:201:1000:0:0:0:5", 53]
(rdb:1) p @senders
{[["2001:1458:201:1000:0:0:0:5", 53], 54764]=>#<Resolv::DNS::Requester::UnconnectedUDP::Sender:0x238048a @host="2001:1458:201:1000:0:0:0:5", @msg="\xD5\xEC\x01\x00\x00\x01\x00\x00\x00\x00\x00\x00\x03web\x04cern\x02ch\x00\x00\x01\x00\x01", @data=#<Resolv::DNS::Name: web.cern.ch.>, @sock=#<UDPSocket:fd 13>, @port=53>}
(rdb:1) c
188.184.9.235
# SUCCESS

nbarrientos · 2016-02-14T11:38:07Z

This could be a candidate for a patch, so the responsibility to compare addresses is delegated to IPAddr (instead of merely comparing strings):

--- lib/ruby/1.9/resolv.rb.orig 2016-02-14 12:16:06.916368212 +0100
+++ lib/ruby/1.9/resolv.rb      2016-02-14 12:30:17.925784003 +0100
@@ -2,6 +2,7 @@
 require 'fcntl'
 require 'timeout'
 require 'thread'
+require 'ipaddr'

 begin
   require 'securerandom'
@@ -728,13 +729,13 @@

         def recv_reply(readable_socks)
           reply, from = readable_socks[0].recvfrom(UDPSize)
-          return reply, [from[3],from[1]]
+          return reply, [IPAddr.new(from[3]),from[1]]
         end

         def sender(msg, data, host, port=Port)
           sock = @socks_hash[host.index(':') ? "::" : "0.0.0.0"]
           return nil if !sock
-          service = [host, port]
+          service = [IPAddr.new(host), port]
           id = DNS.allocate_request_id(host, port)
           request = msg.encode
           request[0,2] = [id].pack('n')

This way it does not matter if the address returned by recvfrom is compressed or expanded, as the one it has to be compared to is also an IPAddr object.

# bin/jruby -r debug -e "require 'resolv'; puts Resolv::DNS.new(:nameserver => ['2001:1458:201:1000::5', '2001:1458:201:1100::5']).getaddress 'web.cern.ch'"
Debug.rb
Emacs support available.
(rdb:1) b /root/jruby-1.7.24/lib/ruby/1.9/resolv.rb:677
Set breakpoint 1 at /root/jruby-1.7.24/lib/ruby/1.9/resolv.rb:677
(rdb:1) c
Breakpoint 1, request at /root/jruby-1.7.24/lib/ruby/1.9/resolv.rb:677
/root/jruby-1.7.24/lib/ruby/1.9/resolv.rb:677:          if s = @senders[[from,msg.id]]
(rdb:1) l
[672, 681] in /root/jruby-1.7.24/lib/ruby/1.9/resolv.rb
   672            begin
   673              msg = Message.decode(reply)
   674            rescue DecodeError
   675              next # broken DNS message ignored
   676            end
=> 677            if s = @senders[[from,msg.id]]
   678              break
   679            else
   680              # unexpected DNS message ignored
   681            end
(rdb:1) p from
[#<IPAddr: IPv6:2001:1458:0201:1000:0000:0000:0000:0005/ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff>, 53]
(rdb:1) p @senders
{[[#<IPAddr: IPv6:2001:1458:0201:1000:0000:0000:0000:0005/ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff>, 53], 30628]=>#<Resolv::DNS::Requester::UnconnectedUDP::Sender:0x68c0dad5 @host="2001:1458:201:1000::5", @msg="w\xA4\x01\x00\x00\x01\x00\x00\x00\x00\x00\x00\x03web\x04cern\x02ch\x00\x00\x01\x00\x01", @data=#<Resolv::DNS::Name: web.cern.ch.>, @sock=#<UDPSocket:fd 13>, @port=53>}
(rdb:1) c
188.184.9.235
#

nbarrientos · 2016-02-14T20:35:18Z

BTW, the IPv6 nameservers used above for all the examples are not accesible from the Internet so expect a ResolvTimeout to be raised anyway, masking the bug. It should be easy to reproduce the problem though using Google's public nameservers (2001:4860:4860::8888 and 2001:4860:4860::8844).

nbarrientos · 2016-02-14T22:16:13Z

Now using Google's nameservers. Slightly different test environment though (OSX, Java8) but same results:

$ host web.cern.ch 2001:4860:4860::8888
Using domain server:
Name: 2001:4860:4860::8888
Address: 2001:4860:4860::8888#53
Aliases:

web.cern.ch has address 188.184.9.235

Unpatched:

$ bin/jruby -v
jruby 1.7.24 (1.9.3p551) 2016-01-20 bd68d85 on Java HotSpot(TM) 64-Bit Server VM 1.8.0_74-b02 +jit [darwin-x86_64]
$ time bin/jruby -e "require 'resolv'; puts Resolv::DNS.new(:nameserver => ['2001:4860:4860::8888', '2001:4860:4860::8844']).getaddress 'web.cern.ch'"
Resolv::ResolvError: DNS result has no information for web.cern.ch
  getaddress at /Users/nacho/Downloads/jruby-1.7.24/lib/ruby/1.9/resolv.rb:373
      (root) at -e:1

real    2m41.820s
user    0m5.326s
sys 0m0.372s

Patched:

$ bin/jruby -v
jruby 1.7.24 (1.9.3p551) 2016-01-20 bd68d85 on Java HotSpot(TM) 64-Bit Server VM 1.8.0_74-b02 +jit [darwin-x86_64]
$ time bin/jruby -e "require 'resolv'; puts Resolv::DNS.new(:nameserver => ['2001:4860:4860::8888', '2001:4860:4860::8844']).getaddress 'web.cern.ch'"
188.184.9.235

real    0m2.007s
user    0m4.815s
sys 0m0.215s

And unpatched again but with uncompressed addresses:

$ time bin/jruby -e "require 'resolv'; puts Resolv::DNS.new(:nameserver => ['2001:4860:4860:0:0:0:0:8888', '2001:4860:4860:0:0:0:0:8844']).getaddress 'web.cern.ch'"
188.184.9.235

real    0m1.644s
user    0m4.680s
sys 0m0.209s

nbarrientos · 2016-03-01T12:52:49Z

Hello?

headius · 2016-03-04T01:53:36Z

Wow, you put a lot of research into this. I've got it in my queue to look at now.

headius · 2016-04-20T20:02:10Z

This didn't make 9.1 but I'll try to make it a priority for 9.1.1.

nbarrientos · 2016-04-20T20:33:02Z

Thanks.

headius · 2016-08-15T21:20:36Z

Ok, this got backburnered for a while, but I'm not quite sure what to do at this point. I need to know...

Does this work on JRuby 9k? It should have the same resolv.rb as MRI and you said MRI works.
Is your patch still appropriate for JRuby 1.7? I believe MRI no longer maintains 1.8.7 or 1.9.3 stdlib, so we can patch that one unilaterally. If we need to patch JRuby 9k, we'll want to work with MRI to upstream that patch.

Sorry this got left for so long!

headius · 2016-11-08T06:28:20Z

No response since August 15...closing for now, but if it's still an issue and you can reproduce it on 9k, we need to get a reproduction script/repo from you.

nbarrientos · 2016-11-08T07:38:24Z

Hi,

Sorry, I missed your update :(

Does this work on JRuby 9k? It should have the same resolv.rb as MRI and you said MRI works.

It does not, as stated on the third comment.

Is your patch still appropriate for JRuby 1.7? I believe MRI no longer maintains 1.8.7 or 1.9.3 stdlib, so we can patch that one unilaterally. If we need to patch JRuby 9k, we'll want to work with MRI to upstream that patch.

It is I believe, however I selfishly only care about 1.7 which is what we have to run as part of Puppetserer 1.x :)

we need to get a reproduction script/repo from you.

The way to reproduce the bug is explained above in detail, you need:

jRuby 1.7 or 9k.
A box with either IPv6 only or dual stack.
Two (or more) IPv6 nameservers in /etc/resolv.conf expressed in the compact form.

Demos (using the latest 9k that I could find):

MRI, compacted IPv6 nameservers: it works.

# cat /etc/resolv.conf 
; generated by /usr/sbin/dhclient-script
search cern.ch.
nameserver 2001:1458:201:1100::5
nameserver 2001:1458:201:1000::5
# ruby -e "require 'resolv'; puts Resolv.getaddress 'web.cern.ch'"
188.184.9.235
# ruby -v
ruby 2.0.0p598 (2014-11-13) [x86_64-linux]

jRuby 9.1.5.0, compacted nameservers, it hangs:

# cat /etc/resolv.conf 
; generated by /usr/sbin/dhclient-script
search cern.ch.
nameserver 2001:1458:201:1100::5
nameserver 2001:1458:201:1000::5
# PATH=$PATH:jruby-9.1.5.0/bin/
# jruby -v
jruby 9.1.5.0 (2.3.1) 2016-09-07 036ce39 OpenJDK 64-Bit Server VM 24.111-b01 on 1.7.0_111-mockbuild_2016_07_27_10_50-b00 +jit [linux-x86_64]
# jruby -e "require 'resolv'; puts Resolv.getaddress 'web.cern.ch'"
.... hangs forever, had to CTRL+C it ...

jRuby 9.1.5.0, expanded nameservers, it works:

# cat /etc/resolv.conf 
; generated by /usr/sbin/dhclient-script
search cern.ch.
;nameserver 2001:1458:201:1100::5
;nameserver 2001:1458:201:1000::5
nameserver 2001:1458:201:1000:0:0:0:5
nameserver 2001:1458:201:1100:0:0:0:5
# jruby -e "require 'resolv'; puts Resolv.getaddress 'web.cern.ch'"
188.184.9.235

And for completeness, jRuby 1.7.26 and compacted nameservers, it hangs:

# cat /etc/resolv.conf 
; generated by /usr/sbin/dhclient-script
search cern.ch.
nameserver 2001:1458:201:1100::5
nameserver 2001:1458:201:1000::5
# PATH=$PATH:jruby-1.7.26/bin/
# jruby -v
jruby 1.7.26 (1.9.3p551) 2016-08-26 69763b8 on OpenJDK 64-Bit Server VM 1.7.0_111-mockbuild_2016_07_27_10_50-b00 +jit [linux-amd64]
# jruby -e "require 'resolv'; puts Resolv.getaddress 'web.cern.ch'"
... hangs...

Those nameservers cannot be used from the Internet, however this should be reproducible using Google's public IPv6 nameservers: 2001:4860:4860::8888 and 2001:4860:4860::8844.

Hope this helps.

nbarrientos · 2016-11-29T18:26:28Z

Could at least the issue be reopened? Thanks.

nbarrientos · 2017-01-13T12:30:22Z

Hi,

Could this issue be reopened, please?

nbarrientos · 2017-02-03T07:53:28Z

@headius ?

When resolving names using compressed IPv6-only DNS servers, it could happen that when processing the response of a name query, the IP of the sender is not compressed so using a simple string comparison is not safe. This patch encapsulates the from address and the address of the sender in IPAddr objects so the comparison is safe and just works in all cases. This fixes jruby#3663.

nbarrientos · 2017-02-19T19:11:57Z

#4496

nbarrientos · 2017-02-19T20:04:05Z

And yes, this is still an issue in jRuby 9.1.7.0.

# grep nameserver /etc/resolv.conf 
nameserver 2001:1458:201:1100::5
nameserver 2001:1458:201:1100::5
# jruby -v
jruby 9.1.7.0 (2.3.1) 2017-01-11 68056ae OpenJDK 64-Bit Server VM 25.121-b13 on 1.8.0_121-b13 +jit [linux-x86_64]
# time jruby -e "require 'resolv'; puts Resolv.getaddress 'web.cern.ch'"
Resolv::ResolvError: no address for web.cern.ch
  getaddress at /root/jruby-9.1.7.0/lib/ruby/stdlib/resolv.rb:100
  getaddress at /root/jruby-9.1.7.0/lib/ruby/stdlib/resolv.rb:50
      <main> at -e:1

real	2m44.461s
user	0m5.107s
sys	0m0.272s
# PATH=jruby-9.1.7.0-patched/bin/:$PATH
## jruby-9.1.7.0-patched contains
## https://patch-diff.githubusercontent.com/raw/jruby/jruby/pull/4496.patch
# jruby -v
jruby 9.1.7.0 (2.3.1) 2017-02-19 fffffff OpenJDK 64-Bit Server VM 25.121-b13 on 1.8.0_121-b13 +jit [linux-x86_64]
# time jruby -e "require 'resolv'; puts Resolv.getaddress 'web.cern.ch'"
188.184.9.235

real	0m4.568s
user	0m4.309s
sys	0m0.212s

headius · 2017-02-23T15:15:20Z

I commented on #4496 but I'll mention again here: I'd be more comfortable patching standard library if we could prove MRI exhibits the same problem.

I also apologize for missing all your activity! Much of your self-conversation here took place while @enebo and I were at FOSDEM and not monitoring issues as closely. We'll get this wrapped up for 9.1.8.0.

nbarrientos · 2017-02-23T16:56:19Z

No problem, thanks anyway for your time 😃

I've just replied on the merge request.

Use ipaddr to compare the sender's address When resolving names using compressed IPv6-only DNS servers, it could happen that when processing the response of a name query, the IP of the sender is not compressed so using a simple string comparison is not safe. This patch encapsulates the from address and the address of the sender in IPAddr objects so the comparison is safe and just works in all cases. This fixes jruby/jruby#3663.

See jruby/jruby#3663.

nbarrientos mentioned this issue Feb 13, 2016

Resolv::DNS.new.getresource makes it hang forever #3568

Open

nbarrientos changed the title ~~Never ending getaddress() call when using compressed IPv6 nameservers in /etc/resolv.conf~~ Never-ending getaddress() call when using compressed IPv6 nameservers in /etc/resolv.conf Feb 13, 2016

headius added this to the JRuby 9.1.1.0 milestone Apr 20, 2016

headius modified the milestones: JRuby 9.1.1.0, JRuby 9.1.2.0 May 11, 2016

enebo modified the milestones: JRuby 9.1.2.0, JRuby 9.1.3.0 May 23, 2016

headius modified the milestones: JRuby 9.1.4.0, JRuby 9.1.3.0 Aug 22, 2016

headius added stdlib JRuby 9000 feedback needed labels Aug 22, 2016

enebo modified the milestones: JRuby 9.1.4.0, JRuby 9.1.5.0 Sep 1, 2016

enebo modified the milestones: JRuby 9.1.4.0, JRuby 9.1.5.0, JRuby 9.1.6.0 Sep 1, 2016

headius closed this as completed Nov 8, 2016

headius modified the milestones: Invalid or Duplicate, JRuby 9.1.6.0 Nov 8, 2016

nbarrientos mentioned this issue Feb 19, 2017

Use ipaddr to compare the sender's address #4496

Merged

nbarrientos mentioned this issue Feb 19, 2017

Socket.getaddrinfo returns wrong data when IPv6 is disabled and running on Java8 #4479

Closed

headius reopened this Feb 22, 2017

headius modified the milestones: JRuby 9.1.8.0, Invalid or Duplicate Feb 22, 2017

headius closed this as completed in #4496 Mar 1, 2017

headius mentioned this issue Mar 1, 2017

Investigate behavioral differences in address lookup #4517

Open

headius added a commit to jruby/ruby that referenced this issue Oct 15, 2019

Restore use of IPAddr for address problems.

92bd479

See jruby/jruby#3663.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Never-ending getaddress() call when using compressed IPv6 nameservers in /etc/resolv.conf #3663

Never-ending getaddress() call when using compressed IPv6 nameservers in /etc/resolv.conf #3663

nbarrientos commented Feb 13, 2016

nbarrientos commented Feb 13, 2016

nbarrientos commented Feb 13, 2016

nbarrientos commented Feb 13, 2016

nbarrientos commented Feb 14, 2016

nbarrientos commented Feb 14, 2016

nbarrientos commented Feb 14, 2016

nbarrientos commented Feb 14, 2016

nbarrientos commented Mar 1, 2016

headius commented Mar 4, 2016

headius commented Apr 20, 2016

nbarrientos commented Apr 20, 2016

headius commented Aug 15, 2016

headius commented Nov 8, 2016

nbarrientos commented Nov 8, 2016

nbarrientos commented Nov 29, 2016

nbarrientos commented Jan 13, 2017

nbarrientos commented Feb 3, 2017

nbarrientos commented Feb 19, 2017

nbarrientos commented Feb 19, 2017

headius commented Feb 23, 2017

nbarrientos commented Feb 23, 2017

Never-ending getaddress() call when using compressed IPv6 nameservers in /etc/resolv.conf #3663

Never-ending getaddress() call when using compressed IPv6 nameservers in /etc/resolv.conf #3663

Comments

nbarrientos commented Feb 13, 2016

nbarrientos commented Feb 13, 2016

nbarrientos commented Feb 13, 2016

nbarrientos commented Feb 13, 2016

nbarrientos commented Feb 14, 2016

nbarrientos commented Feb 14, 2016

nbarrientos commented Feb 14, 2016

nbarrientos commented Feb 14, 2016

nbarrientos commented Mar 1, 2016

headius commented Mar 4, 2016

headius commented Apr 20, 2016

nbarrientos commented Apr 20, 2016

headius commented Aug 15, 2016

headius commented Nov 8, 2016

nbarrientos commented Nov 8, 2016

nbarrientos commented Nov 29, 2016

nbarrientos commented Jan 13, 2017

nbarrientos commented Feb 3, 2017

nbarrientos commented Feb 19, 2017

nbarrientos commented Feb 19, 2017

headius commented Feb 23, 2017

nbarrientos commented Feb 23, 2017