开发者

Simple issue with floats and ints

开发者 https://www.devze.com 2023-02-11 16:05 出处:网络
A simple question best asked with two lines of code: CGFloat answer = (abs(-27.460757f)开发者_StackOverflow社区 - 9.0f) * (800.0f / (46.0f - 9.0f));

A simple question best asked with two lines of code:

CGFloat answer = (abs(-27.460757f)开发者_StackOverflow社区 - 9.0f) * (800.0f / (46.0f - 9.0f));
    NSLog(@"According to my calculator, answer should be 399.15, but instead it is: %f", answer);

When I run this in Xcode (specifically, in the iPhone simulator), I get:

According to my calculator, answer should be 399.15, but instead it is: 389.189209

Is this just due to my lack of understanding of how floats are rounded?

Thanks!

Stewart


The abs() function operates on integers, so abs(-27.460757f) returns 27. Since you’re using floats, use fabsf() instead.

0

精彩评论

暂无评论...
验证码 换一张
取 消