Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AWS SDK Uninitialized Constant Error #3645

Closed
smythey21 opened this issue Feb 4, 2016 · 15 comments
Closed

AWS SDK Uninitialized Constant Error #3645

smythey21 opened this issue Feb 4, 2016 · 15 comments

Comments

@smythey21
Copy link

Using the latest version of aws-sdk, we started receiving the error uninitialized constant Aws::Client::Errors with jruby-9.0.5.0 (though it works fine with 9.0.4.0) when calling .exists? on an s3 resource. Wondering if anything has changed in 9.0.5.0 that would cause this?

[2016-02-04 16:09:23 -0500] [d5ef5747-ebee-40] uninitialized constant Aws::Client::Errors
** [Raven] Event not sent due to excluded environment: development
[2016-02-04 16:09:23 -0500] [d5ef5747-ebee-40] ["/Users/kevinsmyth/.rbenv/versions/jruby-9.0.5.0/lib/ruby/gems/shared/gems/aws-sdk-core-2.2.0/lib/aws-sdk-core/xml/error_handler.rb:25:in `const_get'",
 "/Users/kevinsmyth/.rbenv/versions/jruby-9.0.5.0/lib/ruby/gems/shared/gems/aws-sdk-core-2.2.0/lib/aws-sdk-core/xml/error_handler.rb:25:in `error'",
 "/Users/kevinsmyth/.rbenv/versions/jruby-9.0.5.0/lib/ruby/gems/shared/gems/aws-sdk-core-2.2.0/lib/aws-sdk-core/xml/error_handler.rb:9:in `block in call'",
 "/Users/kevinsmyth/.rbenv/versions/jruby-9.0.5.0/lib/ruby/gems/shared/gems/aws-sdk-core-2.2.0/lib/seahorse/client/response.rb:43:in `call'",
 "/Users/kevinsmyth/.rbenv/versions/jruby-9.0.5.0/lib/ruby/gems/shared/gems/aws-sdk-core-2.2.0/lib/seahorse/client/response.rb:43:in `block in on'",
 "/Users/kevinsmyth/.rbenv/versions/jruby-9.0.5.0/lib/ruby/gems/shared/gems/aws-sdk-core-2.2.0/lib/seahorse/client/http/response.rb:139:in `block in on_success'",
 "/Users/kevinsmyth/.rbenv/versions/jruby-9.0.5.0/lib/ruby/gems/shared/gems/aws-sdk-core-2.2.0/lib/seahorse/client/http/response.rb:166:in `call'",
 "/Users/kevinsmyth/.rbenv/versions/jruby-9.0.5.0/lib/ruby/gems/shared/gems/aws-sdk-core-2.2.0/lib/seahorse/client/http/response.rb:166:in `block in listener'",
 "/Users/kevinsmyth/.rbenv/versions/jruby-9.0.5.0/lib/ruby/gems/shared/gems/aws-sdk-core-2.2.0/lib/seahorse/client/http/response.rb:130:in `call'",
 "/Users/kevinsmyth/.rbenv/versions/jruby-9.0.5.0/lib/ruby/gems/shared/gems/aws-sdk-core-2.2.0/lib/seahorse/client/http/response.rb:130:in `on_done'",
 "/Users/kevinsmyth/.rbenv/versions/jruby-9.0.5.0/lib/ruby/gems/shared/gems/aws-sdk-core-2.2.0/lib/seahorse/client/http/response.rb:137:in `on_success'",
 "/Users/kevinsmyth/.rbenv/versions/jruby-9.0.5.0/lib/ruby/gems/shared/gems/aws-sdk-core-2.2.0/lib/seahorse/client/response.rb:42:in `on'",
 "/Users/kevinsmyth/.rbenv/versions/jruby-9.0.5.0/lib/ruby/gems/shared/gems/aws-sdk-core-2.2.0/lib/aws-sdk-core/xml/error_handler.rb:8:in `call'",
 "/Users/kevinsmyth/.rbenv/versions/jruby-9.0.5.0/lib/ruby/gems/shared/gems/aws-sdk-core-2.2.0/lib/aws-sdk-core/plugins/s3_request_signer.rb:64:in `call'",
 "/Users/kevinsmyth/.rbenv/versions/jruby-9.0.5.0/lib/ruby/gems/shared/gems/aws-sdk-core-2.2.0/lib/aws-sdk-core/plugins/s3_redirects.rb:15:in `call'",
 "/Users/kevinsmyth/.rbenv/versions/jruby-9.0.5.0/lib/ruby/gems/shared/gems/aws-sdk-core-2.2.0/lib/aws-sdk-core/plugins/retry_errors.rb:87:in `call'",
 "/Users/kevinsmyth/.rbenv/versions/jruby-9.0.5.0/lib/ruby/gems/shared/gems/aws-sdk-core-2.2.0/lib/aws-sdk-core/plugins/s3_md5s.rb:33:in `call'",
 "/Users/kevinsmyth/.rbenv/versions/jruby-9.0.5.0/lib/ruby/gems/shared/gems/aws-sdk-core-2.2.0/lib/aws-sdk-core/plugins/s3_expect_100_continue.rb:21:in `call'",
 "/Users/kevinsmyth/.rbenv/versions/jruby-9.0.5.0/lib/ruby/gems/shared/gems/aws-sdk-core-2.2.0/lib/aws-sdk-core/plugins/s3_bucket_dns.rb:31:in `call'",
 "/Users/kevinsmyth/.rbenv/versions/jruby-9.0.5.0/lib/ruby/gems/shared/gems/aws-sdk-core-2.2.0/lib/aws-sdk-core/rest/handler.rb:7:in `call'",
 "/Users/kevinsmyth/.rbenv/versions/jruby-9.0.5.0/lib/ruby/gems/shared/gems/aws-sdk-core-2.2.0/lib/aws-sdk-core/plugins/user_agent.rb:12:in `call'",
 "/Users/kevinsmyth/.rbenv/versions/jruby-9.0.5.0/lib/ruby/gems/shared/gems/aws-sdk-core-2.2.0/lib/seahorse/client/plugins/endpoint.rb:41:in `call'",
 "/Users/kevinsmyth/.rbenv/versions/jruby-9.0.5.0/lib/ruby/gems/shared/gems/aws-sdk-core-2.2.0/lib/aws-sdk-core/plugins/param_validator.rb:21:in `call'",
 "/Users/kevinsmyth/.rbenv/versions/jruby-9.0.5.0/lib/ruby/gems/shared/gems/aws-sdk-core-2.2.0/lib/aws-sdk-core/plugins/s3_sse_cpk.rb:18:in `call'",
 "/Users/kevinsmyth/.rbenv/versions/jruby-9.0.5.0/lib/ruby/gems/shared/gems/aws-sdk-core-2.2.0/lib/aws-sdk-core/plugins/param_converter.rb:20:in `call'",
 "/Users/kevinsmyth/.rbenv/versions/jruby-9.0.5.0/lib/ruby/gems/shared/gems/aws-sdk-core-2.2.0/lib/seahorse/client/plugins/response_target.rb:21:in `call'",
 "/Users/kevinsmyth/.rbenv/versions/jruby-9.0.5.0/lib/ruby/gems/shared/gems/aws-sdk-core-2.2.0/lib/seahorse/client/request.rb:70:in `send_request'",
 "/Users/kevinsmyth/.rbenv/versions/jruby-9.0.5.0/lib/ruby/gems/shared/gems/aws-sdk-core-2.2.0/lib/aws-sdk-core/waiters/poller.rb:61:in `send_request'",
 "/Users/kevinsmyth/.rbenv/versions/jruby-9.0.5.0/lib/ruby/gems/shared/gems/aws-sdk-core-2.2.0/lib/aws-sdk-core/waiters/poller.rb:47:in `call'",
 "/Users/kevinsmyth/.rbenv/versions/jruby-9.0.5.0/lib/ruby/gems/shared/gems/aws-sdk-core-2.2.0/lib/aws-sdk-core/waiters/waiter.rb:104:in `block in poll'",
 "/Users/kevinsmyth/.rbenv/versions/jruby-9.0.5.0/lib/ruby/gems/shared/gems/aws-sdk-core-2.2.0/lib/aws-sdk-core/waiters/waiter.rb:101:in `loop'",
 "/Users/kevinsmyth/.rbenv/versions/jruby-9.0.5.0/lib/ruby/gems/shared/gems/aws-sdk-core-2.2.0/lib/aws-sdk-core/waiters/waiter.rb:101:in `poll'",
 "/Users/kevinsmyth/.rbenv/versions/jruby-9.0.5.0/lib/ruby/gems/shared/gems/aws-sdk-core-2.2.0/lib/aws-sdk-core/waiters/waiter.rb:91:in `block in wait'",
 "/Users/kevinsmyth/.rbenv/versions/jruby-9.0.5.0/lib/ruby/gems/shared/gems/aws-sdk-core-2.2.0/lib/aws-sdk-core/waiters/waiter.rb:90:in `catch'",
 "/Users/kevinsmyth/.rbenv/versions/jruby-9.0.5.0/lib/ruby/gems/shared/gems/aws-sdk-core-2.2.0/lib/aws-sdk-core/waiters/waiter.rb:90:in `block in wait'",
 "/Users/kevinsmyth/.rbenv/versions/jruby-9.0.5.0/lib/ruby/gems/shared/gems/aws-sdk-core-2.2.0/lib/aws-sdk-core/waiters/waiter.rb:89:in `catch'",
 "/Users/kevinsmyth/.rbenv/versions/jruby-9.0.5.0/lib/ruby/gems/shared/gems/aws-sdk-core-2.2.0/lib/aws-sdk-core/waiters/waiter.rb:89:in `wait'",
 "/Users/kevinsmyth/.rbenv/versions/jruby-9.0.5.0/lib/ruby/gems/shared/gems/aws-sdk-core-2.2.0/lib/aws-sdk-core/client_waiters.rb:110:in `wait_until'",
 "/Users/kevinsmyth/.rbenv/versions/jruby-9.0.5.0/lib/ruby/gems/shared/gems/aws-sdk-resources-2.2.0/lib/aws-sdk-resources/operations.rb:214:in `call'",
 "/Users/kevinsmyth/.rbenv/versions/jruby-9.0.5.0/lib/ruby/gems/shared/gems/aws-sdk-resources-2.2.0/lib/aws-sdk-resources/operation_methods.rb:19:in `block in wait_until_exists'",
 "/Users/kevinsmyth/.rbenv/versions/jruby-9.0.5.0/lib/ruby/gems/shared/gems/aws-sdk-resources-2.2.0/lib/aws-sdk-resources/resource.rb:131:in `exists?'",
@torrancew
Copy link

@smythey21 I see the aws-sdk README notes that this is a common error when unintentionally upgrading from v1.x to v2.x of their API. Is it possible you're impacted by that?

Edit Seems that that's the opposite error message from what is presented here (v1 used AWS, v2 Aws)

@smythey21
Copy link
Author

@torrancew we are using aws-sdk 2.2.13 and jruby 9.0.4.0 in production with no issues, but thanks for pointing that out :)

@jonp
Copy link

jonp commented Feb 9, 2016

@smythey21 I'm getting the same thing trying to use S3.

It seems that the call to const_set in the aws-sdk-core gem doesn't sit well with jruby 9. In old jruby 1.7, that would actually set the const in the module and Aws::S3 would correctly think that its name was "Aws::S3". In jruby 9, however, it doesn't seem to "take" in that you can still use Aws::S3, but it thinks that its name is a module instance (see below). So, in the line above your error the aws-sdk-core gem is (a bit sloppily) trying to do some tricks with the name of the module to try to decide what handler to use for the error, only to have an unexpected name to parse. It was expecting something like Aws::S3::Client, from which it wants to pull "S3", but instead gets something like #<#<Module:0x9addea4>::Client:0x5341eab4>, from which it pulls "Client."

So, there are two bugs: first: jruby's const_set is acting odd in this case and second: the aws gem is playing fast and loose with names.

The simple repro for this is:

» irb
jruby-9.0.5.0 :001 > require 'aws-sdk'; Aws::S3
 => #<Module:0x4ff4357f>

The return should be Aws::S3 and not #<Module:0x4ff4357f>.

The good news is that a second call to const_set does seem to work, so if you tuck this line in your app's startup somewhere after where you require 'aws-sdk' you'll have a workaround:

module Aws; const_set(:S3, Aws::S3); end # or whichever AWS service you're trying to use

Unfortunately, I can't seem to repro this without using the aws-sdk gem, so I can't offer conclusive proof that it's really jruby's fault. OTOH, the fact that it behaves correctly on jruby-1.7 is a strong indicator.

@headius
Copy link
Member

headius commented Feb 13, 2016

Wow, this is really weird. It works fine in a simple script:

$ jruby -e 'module Foo; m = Module.new { extend Enumerable }; const_set :Bar, m; end; p Foo::Bar'
Foo::Bar

But I can confirm the const_set in aws-sdk-core.rb in add_service does not cause it to be set.

This is likely related to another bug report that I can't find right now.

After some investigation, it appears that 02efc4d by @kares broke this. Many paths now cause constants to be defined as "hidden", which skips setting their names.

@kares Can you look into this and repair your patch?

@headius
Copy link
Member

headius commented Feb 13, 2016

I'm rather surprised this didn't regress any tests. We need to add some ruby specs for this.

@kares
Copy link
Member

kares commented Feb 13, 2016

yy, can look into this next week (unless its fixed by than). thanks.

@headius
Copy link
Member

headius commented Feb 14, 2016

@kares I may have a look at it during my flights home today. Can you point me toward any tests you used to verify your fixes, so I don't regress anything?

@kares
Copy link
Member

kares commented Feb 14, 2016

did it around constant "hiding" in Java int (e.g. private inner classes) thus spec:ji should be green ...
mostly needed hidden flag to be part of the const storing API (have some local unfinished JI tunings that consume it) ... if all specs/tests pass do not worry about changing anything else.

@headius
Copy link
Member

headius commented Feb 14, 2016

@kares Ok.

@kares
Copy link
Member

kares commented Feb 16, 2016

the issue relates to autoloading ... able to reproduce in a small script. what seems specific here is the auto-loading part doing a callback :

  module GH3645
    class << self

      def add_mod(name) # call this from autoloaded script
        mod = Module.new { extend GH3645 }
        const_set(name, mod)
        mod
      end

    end
    autoload :S3, File.expand_path('GH-3645_autoload', File.dirname(__FILE__))
  end

@kares
Copy link
Member

kares commented Feb 16, 2016

OK, thing I grasp what's going on. the regression is related (and expected) due previous autoload logic.
regression came in with fixing autoload not to load on const_defined? checks b216c5d ... previously the const_set would always initialize the autoloaded constant "early" and thus set its parent module, whether after the change it does not since there's logic to wait for autoload-ing to finish loading the file. the "finish" autoload logic simply swaps the constant in the table.

I believe the side effect of the previous logic to set the parent (and name) early on was better than to add it to finishAutoload ... but will look into MRI's test suite for more on how/whether this is tested.

@headius
Copy link
Member

headius commented Feb 16, 2016

@kares Ok I'll let you take it from here. I did try the patching setConstant to default to non-hidden, which didn't break any JI tests (did you add some for the hidden stuff?)...but it also did not fix the issue.

kares added a commit that referenced this issue Feb 16, 2016
... previously this worked as a side effect of fetchConstant "loading" auto-loads
but since (b216c5d) `autoload` has been fixed to not initialize prematurely
kares added a commit that referenced this issue Feb 16, 2016
... this is not crucial but restores the same behavior as in JRuby < 9.0.5.0
@kares
Copy link
Member

kares commented Feb 16, 2016

@headius yes there's a spec testing hiding non public inner classes as constants
... as noted this is not related to the commit you linked but a side effect of another (that is why it did not fix).

now fixed and expected to behave as before.

@kares kares closed this as completed Feb 16, 2016
@headius
Copy link
Member

headius commented Feb 16, 2016

@kares Excellent, thank you!

@dcrec1
Copy link

dcrec1 commented Nov 17, 2016

@jonp solution worked for me.

mhoroschun pushed a commit to spinify/logstash-input-cloudwatch-logs that referenced this issue Feb 3, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

6 participants