开发者

Ruby number precision with simple arithmetic

开发者 https://www.devze.com 2023-02-19 17:09 出处:网络
I\'m learning Ruby for fun, and for creating websites also (but that\'s irrelevant). While playing with it, i noticed something \"weird\"

I'm learning Ruby for fun, and for creating websites also (but that's irrelevant). While playing with it, i noticed something "weird"

When I compute 4.21 + 5 with irb, it answers 9.21 (weird, right?)

when I compute 4.23 + 5, it gives 9.23 (waw, that's definitely weird).

and when i type 4.22 + 5, it answers 9.21999... (w...wait! that's really weird).

Hence my question: what's going on? I'd understand this behavior with division or really big numbers, but in this simple case....???

Does it mean that i can't develop an accounting app with Ruby? Is there a patch or some开发者_开发问答thing to be applied? (to my brains, most likely)


You should read up on floating point representation in computers. This guide is a good place to start. Here's the short of it:

Because internally, computers use a format (binary floating-point) that cannot accurately represent a number like 0.1, 0.2 or 0.3 at all.

When the code is compiled or interpreted, your “0.1” is already rounded to the nearest number in that format, which results in a small rounding error even before the calculation happens.

By the way, I'm not sure why you think 4.21 + 5 = 9.21 or 4.23 + 5 = 9.23 is weird. If you think it's weird because one literal is an integer and one is a float, which ends up as a float response, that's how Ruby and some other languages handle differences in number types. It wouldn't be handy if Ruby dropped your float's precision and gave you just an integer back (9), so any integer literals are effectively changed to floats.

As for using floating-point numbers in a financial application, check out Developing financial application. The main takeaway is use the Decimal object vs. Float.


http://ruby-decimal.rubyforge.org/

This is the package you'd want to do reliable Floating Point maths in ruby. The Developint Financial Applications link doesn't reference it, but it makes the important point of using Decimal data types in your database.

Just noting this since it was non obvious for me :)

0

精彩评论

暂无评论...
验证码 换一张
取 消

关注公众号