开发者

Inconsistent rounding in Ruby?

开发者 https://www.devze.com 2022-12-15 03:31 出处:网络
Does Ruby have a bug in its rounding? Why does it behave like this: >> [1.14, 1.15, 1.16].map{|x| \"%.1f\" % x}

Does Ruby have a bug in its rounding? Why does it behave like this:

>> [1.14, 1.15, 1.16].map{|x| "%.1f" % x}
=> ["1.1", "1.1", "1.2"]
>> [开发者_运维问答1.4, 1.5, 1.6].map{|x| "%.0f" % x}
=> ["1", "2", "2"]

as in, why does 1.15 get rounded to 1.1, but 1.5 gets rounded to 2? At the very least, isn't this inconsistent? the behaviour is the same in ruby 1.9.1 and ruby 1.8.7.


Take a look at my answer to this question

Why does Perl's sprintf not round floating point numbers correctly?

This may be the same thing


You're using floating point numbers. Floating point numbers aren't precise. See http://en.wikipedia.org/wiki/IEEE_754-2008 for an introduction in the standard.

The short version is: NEVER use floats for anything where you need precision in any way!


It's useful to recall and also quite ironic to contemplate, but floating point numbers only represent exactly: (a) a few fractions or (b) all integers.

So, to have an exact representation a fraction must be composed of (negative) powers of two. So, the following fractions are the only ones between .01 and .99 that are exactly represented:

0.25
0.50
0.75

In other words, FP is perfectly accurate when dealing with integers. Go figure.

0

精彩评论

暂无评论...
验证码 换一张
取 消

关注公众号