Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Can't load many files #1591

Closed
rosenfeld opened this issue Mar 28, 2014 · 4 comments
Closed

Can't load many files #1591

rosenfeld opened this issue Mar 28, 2014 · 4 comments
Milestone
Invalid or Duplicate

Comments

@rosenfeld
Copy link

I was having an interesting discussion with Myron Marston about autoload and about how fast require is, and while trying to perform my tests with JRuby I got this error:

$ ruby -I . -r benchmark -e "puts Benchmark.measure{require 'main'}.total"
LoadError: load error: /home/rodrigo/tmp/require-test/lib/rb -- java.lang.OutOfMemoryError: GC overhead limit exceeded
           require at org/jruby/RubyKernel.java:1085
           require at /home/rodrigo/.rvm/rubies/jruby-1.7.11/lib/ruby/shared/rubygems/core_ext/kernel_require.rb:55
           require at /home/rodrigo/.rvm/rubies/jruby-1.7.11/lib/ruby/shared/rubygems/core_ext/kernel_require.rb:53
  require_relative at file:/home/rodrigo/.rvm/rubies/jruby-1.7.11/lib/jruby.jar!/jruby/kernel19/kernel.rb:21
            (root) at /home/rodrigo/tmp/require-test/main.rb:444
           require at org/jruby/RubyKernel.java:1085
            (root) at /home/rodrigo/.rvm/rubies/jruby-1.7.11/lib/ruby/shared/rubygems/core_ext/kernel_require.rb:1
            (root) at -e:1
           require at /home/rodrigo/.rvm/rubies/jruby-1.7.11/lib/ruby/shared/rubygems/core_ext/kernel_require.rb:55
            (root) at -e:1

The original discussion is here:
rspec/rspec-rails#975

The code that generates the lib files is this:

require 'fileutils'
FileUtils.mkdir_p 'lib'

body_content = (('aa'..'zz').map do |l|
  "
  def instance_method_#{l}
    'anything you want'
  end
  def self.class_method_#{l}
    'class method return value'
  end
  "
end).join "\n"
('AA'..'ZZ').each do |l|
  d = l.downcase
  File.write "lib/#{d}.rb",
    "class #{l}1
       #{body_content}
     end"
end

main_content = []
('aa'..'zz').each do |l|
  main_content << "require_relative 'lib/#{l}'"
end

File.write 'main.rb', main_content.join("\n")

MRI runs it just fine in about 6s in my computer.
@ratnikov
Copy link
Contributor

It seems like this would be creating about 676 classes. Do you run out of memory if you don't do any class creation?

How much memory do you allot to your jvm?

@rosenfeld
Copy link
Author

I was trying on the JVM 8 so I don't think this is related to memory allocated on the jvm. I haven't tried yet to load the files without creating classes...

@rosenfeld
Copy link
Author

Sorry, it seems jruby uses the information in $JAVA_HOME when available to set which Java to run instead of which java is on the path, so java -version was 1.8 but jruby -version was running on 1.7 because JAVA_HOME was pointing to 1.7 since Grails doesn't support 1.8 yet.

But on 1.8 it just took longer to get to the same results:

$ ruby --version
jruby 1.7.12 (2.0.0p195) 2014-04-15 643e292 on Java HotSpot(TM) 64-Bit Server VM 1.8.0_05-b13 +indy [linux-amd64]
11:39:02 rodrigo@rodrigo:~/tmp
$ ruby main.rb 
LoadError: load error: /home/rodrigo/tmp/lib/pu -- java.lang.OutOfMemoryError: Java heap space
           require at org/jruby/RubyKernel.java:1065
           require at /home/rodrigo/.rvm/rubies/jruby-1.7.12/lib/ruby/shared/rubygems/core_ext/kernel_require.rb:55
           require at /home/rodrigo/.rvm/rubies/jruby-1.7.12/lib/ruby/shared/rubygems/core_ext/kernel_require.rb:53
  require_relative at file:/home/rodrigo/.rvm/rubies/jruby-1.7.12/lib/jruby.jar!/jruby/kernel19/kernel.rb:21
            (root) at main.rb:411

With MRI it just takes 13s:

$ ruby --version
ruby 2.1.1p76 (2014-02-24 revision 45161) [x86_64-linux]
11:45:18 rodrigo@rodrigo:~/tmp
$ time ruby main.rb 

real    0m12.979s
user    0m10.836s
sys     0m0.380s

@kares
Copy link
Member

kares commented Feb 18, 2016

must have been related to a leak ... on recent JRuby works fine, just tried master and got :

real    0m2.253s
user    0m2.884s
sys 0m0.644s

@kares kares closed this as completed Feb 18, 2016
@kares kares added this to the Invalid or Duplicate milestone Feb 18, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
Invalid or Duplicate
Development

No branches or pull requests
3 participants
@ratnikov @rosenfeld @kares