开发者

Odd behavior with NSUInteger - can't convert to float properly

开发者 https://www.devze.com 2023-01-17 14:27 出处:网络
Here is my situation. Its driving me nuts: I have an NSMutableArray with a count value of 开发者_如何学Python517. I have a double value that is my multiplier.

Here is my situation. Its driving me nuts:

I have an NSMutableArray with a count value of 开发者_如何学Python517. I have a double value that is my multiplier.

double multiplier = 0.1223;
double result = [myArray count] * multiplier; // 63 even (wrong!)

In fact it should be 63.2291. If I go:

double result = [myArray count] * 0.1223; // 63.2291 (right!)

or..

double result = 517 * multiplier; // 63.2291 (right!)

Does this make any sense to anyone?

Addendum:

here is my actual function:

- (double) getValueForPercentage:(double)percVal
{
   int adjustedCount = [originalData count] - 1;
   double final = percVal * (double)adjustedCount;
   return final;
}

I never get any digits beyond the decimal point when I do this. It does however work if I get rid of the "-1", a-la:

- (double) getValueForPercentage:(double)percVal
{
   int adjustedCount = [originalData count];
   double final = percVal * (double)adjustedCount;
   return final;
}

Of course, I need to have the -1.

Second addendum:

Another interesting thing I noted was, if I pass a hard-coded number to this function it works fine, but if I pass the double value that I need to use, it fails:

int pointCount = [srcData getDayCount];

for (int i = 0; i < pointCount; i++) {
   double progress = (double)i/(double)(pointCount - 1);
   double satv = [srcData getValueForPercentage:progress];
   // satv is always a number without any digits beyond the decimal
}


Well, when I started to have these issues i looked around a bit and found no reason or explanation.

What I do now is make everything become an NSNumber and then call doubleValue on it. This should yield the results you're looking for:

NSNumber * pointCount = [NSNumber numberWithUnsignedInt: [srcData getDayCount]];
for (NSInteger i = 0; i < [pointCount intValue]; i++) {
    NSNumber * count = [ NSNumber numberWithInt: i ];
    double progress = [count doubleValue]/[pointCount doubleValue] - 1.0;
    double satv = [srcData getValueForPercentage:progress];
    // satv is always a number without any digits beyond the decimal
}

Hope it helps.

0

精彩评论

暂无评论...
验证码 换一张
取 消

关注公众号