I have been searching the internet trying to understand in its simplist form why this behavior happens.
开发者_运维问答Dim mysingle As Single = 456.11
Dim mybool As Boolean = mysingle = 456.11
In the lines above mybool becomes false. I found this behavior when putting the single into a double I found extra digits showing. The .net documentations states a single is an approimate value :S
I gatehr a single is a 32bit floating point number? But why are extra digits appearing when I have explicitly said what the number is.. surely the memory should store that numbers either side of my number are 0 to fill up the memory location?!
My brain is fried on this one :(
The value being compared in the second statement isn't considered a Single
, rather it's being treated as a Double
. Since you're using VB.NET you can suffix it with a !
to force it to a Single
and this will return True
:
Dim mysingle As Single = 456.11
Dim mybool As Boolean = mysingle = 456.11!
I recommend reading: What Every Computer Scientist Should Know About Floating-Point Arithmetic.
It explains the issues with precision in floating point math in detail. For a simpler variation on the above, see The Floating Point Guide.
Another good reference for understanding floating-point in .NET: https://csharpindepth.com/articles/FloatingPoint
Let me highlight his point on comparisons:
One consequence of all of this is that you should very, very rarely be comparing binary floating point numbers for equality directly. [...] One simple way of doing this is to subtract one from the other, use Math.Abs to find the absolute value of the difference, and then check whether this is lower than a certain tolerance level.
精彩评论