Can anyone explain this weirdness:
Dim result as single = 0
result = CType("8.01", Single) * 100 ' result=801.0 as expected
result = CType("8.02", Single) * 100 ' re开发者_高级运维sult=802.000061 --- not expected
Further to the above
result = 8.02 * 100 ' result = 802.0 as expected
Single
(or float
) only has seven significant digits, so everything that might be printed beyond those is pretty much bogus anyway. This is a normal artifact of floating-point arithmetic. You can only rely on up to 4 digits after the decimal point with your number (as three significant digits are already before the decimal point).
Further elaborated, the numbers 8.01 and 8.02 can't be represented exactly using binary storage (as neither 0.01 nor 0.02 can be represented exactly with sums of fractions of the form 1/2n). The details may vary from number to number but you may see extraneous digits outside the normal precision range when using such numbers.
This doesn't affect the fact that 801 and 802 can be represented exactly but you have no exact number to start with in this case.
ETA: In fact, what you see when you include the calculation directly is just this: The compiler will evaluate the calculation for you and simply write 802
into the program. You can use Reflector to verify that. Also floating point literals in the source code are likely Double
by default so you have much greater precision to begin with here. If result
is a Single
this will be downcasted to Single
and the error in the 16th digit after the decimal point is simply thrown away since it can't fit into Single
anyway.
This is due to a limitation in the way floating point numbers are represented in memory. Have a read through the wikipedia page for the full lowdown - it's not an error, or something specific to .net.
EDIT: these two articles are worth a look too (the latter is math heavy).
- http://www.techradar.com/news/computing/why-computers-suck-at-maths-644771
- http://docs.sun.com/source/806-3568/ncg_goldberg.html
If what you need is to compare floating point values, see here: Comparing floating point values
Single
and Double
in .NET are represented as floating point numbers and unfortunately storing these kinds of numbers is not precise. You will get small discrepancies when working with 'Single' and 'Double' numbers.
However there is an alternative - .NET has a Decimal
type which is much better at dealing with these numbers. It's still not perfect (see http://www.yoda.arachsys.com/csharp/decimal.html - in particular the last paragraph) but will be much better at representing quantities like money.
精彩评论