开发者

Why [NSDecimalNumber longLongValue] return min value

开发者 https://www.devze.com 2023-02-15 16:00 出处:网络
NSDecimalNumber *dn = [[[NSDecimalNumber alloc] initWithString:@\"9223372036854775806\"] autorelease];
NSDecimalNumber *dn = [[[NSDecimalNumber alloc] initWithString:@"9223372036854775806"] autorelease];
long long llVal = [dn longLongValue]; 

why llVal is -9223372036854775808 ?

NSDecimalNumber extends NSNumber so it supposes to handle long long type. doesn't 开发者_如何学JAVAit?


I believe you are running into a small base10->base2 rounding error in -longLong. I would open a defect at bugreport.apple.com. You should be able to round-trip a number within the long long range.

If you stay entirely in NSDecimalNumber, you will note that it does not suffer the rounding error. Here is some code that I think shows the problem very clearly:

unsigned long long longlong = 9223372036854775806;
NSLog(@"longlong: %lu", longlong);
NSDecimalNumber *dn = [NSDecimalNumber decimalNumberWithMantissa:longlong exponent:0 isNegative:NO];
NSLog(@"dn-dn: %@", dn);    // Round-trips fine
NSLog(@"dn+10: %@", [dn decimalNumberByAdding:[NSDecimalNumber decimalNumberWithString:@"10"]]);    // Even does math
NSLog(@"dn-lu: %lu", [dn unsignedLongValue]); // has rounding error

2011-03-07 23:56:15.132 Untitled[16059:a0f] longlong: 9223372036854775806
2011-03-07 23:56:15.135 Untitled[16059:a0f] dn-dn: 9223372036854775806
2011-03-07 23:56:15.135 Untitled[16059:a0f] dn+10: 9223372036854775816
2011-03-07 23:56:15.136 Untitled[16059:a0f] dn-lu: 9223372036854775808


I think its because longLongValue method can be used at NSString and NSNumber and not on NSDecimalNumber.

I dont know it for sure but i think thats the reason the value of your variable is always  -9223372036854775808 even if you change the string on dn

0

精彩评论

暂无评论...
验证码 换一张
取 消

关注公众号