开发者

Round to nearest low-level implementation

开发者 https://www.devze.com 2023-03-03 18:47 出处:网络
I\'ve got some low-level question: Let\'s assume we\'ve got the decimal number 0.1 (we can represent this very exactly by writing 1/10 - but that is not a solution). Converted into a binary numbe开发者

I've got some low-level question:

Let's assume we've got the decimal number 0.1 (we can represent this very exactly by writing 1/10 - but that is not a solution). Converted into a binary numbe开发者_如何学Pythonr, this gives us 0,00011 with the last 4 digits infinitely repeated (0,0001100110011....).

So now assume we want to put this number into a floating point representation, e.g. IEEE 754 single precision and we want to round it so that the nearest possible representation, which gives us

1 0111 1110 0101 0101 0101 0101 0101 011

(fyi: the last digit should be a 0 instead of a 1 if we would simply truncate)

Now, my question is: How to achieve this in software? After all we can't simply use some

x=((x_floor-0.1)<(x_ceil-0.1))?x_floor:x_ceil;

as this would imply a "perfect" 0.1 which is the core of my problem. Any ideas?


I think I've got some answer on my own question: You can decide by checking the following bit of the conversation. If it would be set, you can round to the ceil, because view the following example: we want to round the binary number 0.1001 to 3 digits after the dot. if we round it to 0.100, we would make an error of 0,0001 as well as in the case we round it to 0,101.

But more interesing: if the number to round is 0,1001...1 - the error we would make if we round to 0.100 would be bigger than the error when rounding to 0.101 in any case! This "proof" can be done in a similar way for the case of an unset bit following the actual conversation

0

精彩评论

暂无评论...
验证码 换一张
取 消