Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Garbage Collection not working #3503

Open
stockf opened this issue Nov 29, 2015 · 8 comments
Open

Garbage Collection not working #3503

stockf opened this issue Nov 29, 2015 · 8 comments
Labels
Milestone

Comments

@stockf
Copy link

stockf commented Nov 29, 2015

As described here: http://stackoverflow.com/questions/33824399/jruby-outofmemoryerror-very-small-example?noredirect=1#comment55640031_33824399

run it with JAVA_MEM=-Xmx500m ruby -w -J-verbose:gc to limit the memory to 500 MB and show the gc output.

(1..5_000_000).to_a
(1..5_000_000).to_a

This works fine. However, running this:

(1..5_000_000).to_a
(1..5_000_000).to_a
(1..5_000_000).to_a

I get an OutOfMemoryError (see below for full OutOfMemoryError stack trace)

when allocating memory to the same variable again and again, it does not seem to be garbage collected and eventually throws an OutOfMemory error or Error: Your application used more memory than the safety cap of 500M.

This bug is present in jruby 9.0.4.0 and 9.0.1.0 but not in jruby 1.7.23. (I did not check other versions).

@enebo enebo added the ir label Dec 1, 2015
@enebo enebo added this to the JRuby 9.0.5.0 milestone Dec 1, 2015
@enebo
Copy link
Member

enebo commented Dec 1, 2015

@stockf without looking I am betting we store the temporary variable of the created arrays and since we never leave this scope those temps have not been cleared. If we fully compile these scripts I suspect those temps will be detected to be unused and eliminated (and suddenly the leak will disappear due to DCE running). It is only an educated guess (and noted for first person who looks into this. Speculatively marking as IR bug...

@chrisseaton
Copy link
Contributor

when allocating memory to the same variable

I don't see any variables in your code - do you assign these arrays to anything?

@stockf
Copy link
Author

stockf commented Dec 15, 2015

@chrisseaton you're right, in the above example I didn't even assign it to a variable. The result is the same, though, if you put x = at the beginning of each line.

@chrisseaton
Copy link
Contributor

I does make sense that the array is being stored in a temporary, as @enebo says. This is a common problem in compilers - you innocently store some value somewhere and forget to clear it, only to find it's hanging on to the entire world.

@headius
Copy link
Member

headius commented Jan 19, 2016

A small improvement here might be to not store temporary values if they're never read; This might happen via DCE or it might not.

This is definitely not something we're going to fix in 9.0.5.0 timeframe, though, so I'm punting.

@headius headius modified the milestones: JRuby 9.1.0.0, JRuby 9.0.5.0 Jan 19, 2016
@headius
Copy link
Member

headius commented Jan 19, 2016

@subbuss Your input would be helpful here. Can we make result-generating instructions not actually store anything if their results are never read?

@subbuss
Copy link
Contributor

subbuss commented Jan 20, 2016

Yes, this is a problem with the references being held in the tmp-var array. This is primarily a problem with the 'startup interpreter' which runs code right after building the IR.

One way to fix this is if we have information (via AST @ IR build time) about output of statements aren't used. If known, then for the IR instructions corresponding to those AST nodes, the tmp vars (these will be tmp vars if the result is dead) used as the instruction result can be marked dead which can be used by the interpreter to skip the store.

@headius headius modified the milestones: JRuby 9.1.1.0, JRuby 9.1.0.0 Apr 15, 2016
@headius headius modified the milestones: JRuby 9.1.1.0, JRuby 9.1.2.0 May 11, 2016
@enebo enebo modified the milestones: JRuby 9.1.2.0, JRuby 9.1.3.0 May 23, 2016
@headius headius modified the milestones: JRuby 9.2.0.0, JRuby 9.1.3.0 Aug 22, 2016
@enebo enebo modified the milestones: JRuby 9.2.0.0, JRuby 9.2.1.0 May 15, 2018
@headius headius modified the milestones: JRuby 9.2.1.0, JRuby 9.3.0.0 Oct 11, 2018
@enebo enebo modified the milestones: JRuby 9.3.0.0, Post-9000 May 25, 2021
@enebo
Copy link
Member

enebo commented May 25, 2021

This is an issue but one which requires a lot of work and we have a lot to do. To effectively untargetting it from near term work but not to the point that it is totally untargetted.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

5 participants