开发者

flash as3 rounding numbers automatically

开发者 https://www.devze.com 2023-04-07 17:42 出处:网络
I wrote out a long post trying to explain the exact details of the problem im having, but instead i think ill keep it simple, and ask an example question here:

I wrote out a long post trying to explain the exact details of the problem im having, but instead i think ill keep it simple, and ask an example question here:

var n1:Number = 9.99999999999999;
n1 += 0.000000000000009;
var n2:Number = n1 + 10;
var n3:Number = n1 - 10;

Long story short, n1 = 9.99....7, n2 = 20, n3 = 10.

If i try to make a comparison between n1 and n3, they should be the same but they arn't. I dont care if flash rounds it or not, i just need them to be the same (and they arn't, cause flash rounds in one case, and not the other).

Is there some standard so开发者_如何学Pythonlution for a problem like this?

P.S. I dont need this precision on my numbers, but i also would not like to micromanage the rounding of the numbers EVERY time i do a manipulation (that seems like it could add a LOT of code to the mix). If this is the only solution however, i guess ill just have to do a lot of rounding throughout the code, ha.


The Number in flash is a double precision floating point number. Read more here about them. These "problems" are not unique to flash, but just have to do with how these numbers are stored.

There are a couple of options. Here is a quick little library for fuzzy comparing numbers, within a certain margin of error. Another option would be implement a fixed point math library.


Is it a problem to just wrap in int() if you're trying to compare?

trace(int(n1) == int(n3));
0

精彩评论

暂无评论...
验证码 换一张
取 消

关注公众号