Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[bigdecimal] Loss of precision with different execution order #2524

Closed
emboss opened this issue Jan 27, 2015 · 7 comments
Closed

[bigdecimal] Loss of precision with different execution order #2524

emboss opened this issue Jan 27, 2015 · 7 comments
Labels
JRuby 1.7.x JRuby 9000
Milestone
JRuby 9.0.0.0.pre2

Comments

@emboss
Copy link

emboss commented Jan 27, 2015

Depending on the order of calculation I get different results that should be equal:

require 'bigdecimal'

fraction = BigDecimal.new("0.0095") / 365 * BigDecimal.new(50_000)
# => #<BigDecimal:4f654cee,'0.130136986301369863013698630136986301369863013698630136986301369863013698630136986301369863013698630136986301369863013698630136986301369863013698630136986301369863013698630136986301369863013698630136985E1',201(208)>
r1 = fraction * BigDecimal.new(50_000) / BigDecimal.new(100_000)
# => #<BigDecimal:7134b8a7,'0.65068493150684931506849315068493150684931506849315068493150684931506849315068493150684931506849315068493150684931506849315068493150684931506849315068493150684931506849315068493150684931506849315068493E0',200(204)>
r2 = fraction * (BigDecimal.new(50_000) / BigDecimal.new(100_000)
# => #<BigDecimal:441fbe89,'0.650684931506849315068493150684931506849315068493150684931506849315068493150684931506849315068493150684931506849315068493150684931506849315068493150684931506849315068493150684931506849315068493150684925E0',201(208)>
r1 == r2
# => false

I could reproduce this with both

$ jruby -v
jruby 9.0.0.0.pre1 (2.2.0p0) 2015-01-20 d537cab Java HotSpot(TM) 64-Bit Server VM 25.0-b70 on 1.8.0-b132 +jit [linux-amd64]

and

$ jruby -v
jruby 1.7.18 (1.9.3p551) 2014-12-22 625381c on Java HotSpot(TM) 64-Bit Server VM 1.7.0_67-b01 +jit [linux-amd64]

Thanks for your great work on 9000 btw!
@Ch4s3
Copy link

Ch4s3 commented Jan 27, 2015

That looks like an error with how many decimal places are being stored.

I tried:

r3 = r1.round(200)
 => #<BigDecimal:761257ea,'0.65068493150684931506849315068493150684931506849315068493150684931506849315068493150684931506849315068493150684931506849315068493150684931506849315068493150684931506849315068493150684931506849315068493E0',200(204)>
r4 = r2.round(200)
 => #<BigDecimal:acaf0ac,'0.65068493150684931506849315068493150684931506849315068493150684931506849315068493150684931506849315068493150684931506849315068493150684931506849315068493150684931506849315068493150684931506849315068493E0',200(204)>
r3 == r4
 => true

The 201st decimal place seems to be where it breaks.

k77ch7 added a commit to k77ch7/jruby that referenced this issue Jan 29, 2015
@k77ch7
fix jrubyGH-2524 on master: proper algorithm to set the precision
93c628d
@k77ch7 k77ch7 mentioned this issue Jan 29, 2015
Merged
@enebo enebo closed this as completed in 8fec7f4 Jan 30, 2015
@emboss
Copy link
Author

emboss commented Jan 30, 2015

Thank you, awesome!

@enebo enebo added this to the 9.0.0.0.pre2 milestone Jan 30, 2015
@k77ch7
Copy link
Contributor

k77ch7 commented Feb 1, 2015

@enebo this still doesn't work on current jruby-1_7. Should I open a different PR against jruby-1_7? Thank you.

@enebo
Copy link
Member

enebo commented Feb 2, 2015

@k77ch7 If you are unable to make a new PR I can try and cherry-pick this but it is simpler if you make a new PR

@enebo
Copy link
Member

enebo commented Feb 2, 2015

@k77ch7 1.9+ bugs are a little simpler for us to merge if the PR is submitted to jruby-1_7 first.

k77ch7 added a commit to k77ch7/jruby that referenced this issue Feb 3, 2015
@k77ch7
fix jrubyGH-2524 on jruby-1_7: improved algorithm to set the precision
b3342e7
@k77ch7 k77ch7 mentioned this issue Feb 3, 2015
Merged
@k77ch7
Copy link
Contributor

k77ch7 commented Feb 3, 2015

@enebo I just submitted the PR #2554 to jruby-1_7.

@k77ch7
Copy link
Contributor

k77ch7 commented Feb 3, 2015

@enebo Thanks for your quick response.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
JRuby 1.7.x JRuby 9000
Projects
None yet
Milestone
JRuby 9.0.0.0.pre2
Development

No branches or pull requests
5 participants
@enebo @rtyler @emboss @Ch4s3 @k77ch7