precision

相关标签：javascript jquery android 多少钱 iPhone

How can I make sure a float will always be rounded up with PHP?

I want to make sure a float in PHP is rounded up if any decimal is present, without worrying about mathematical rounding rules. T开发者_如何学Gohis function would work as follows:[详细]

2023-02-12 18:10 分类：问答
How to convert to double with 2 precision - string after dot?

I want to convert this string: 0.55000开发者_C百科000000000004 to this double: 0.55. How to do that?you can use this code to reduce precision part:[详细]

2023-02-11 00:01 分类：问答
How to calculate exact value of a trig function?

I am writing a \"triangle solver\" app for Android, and I was wondering if it would be possible to implement exact values for trig ratios and radian measures. For example, 90 degrees would be output a[详细]

2023-02-10 02:27 分类：问答
Newton's method with specified digits of precision

I\'m trying to write a function in Java that calculates the n-th root of a number. I\'m using Newton\'s method for this. However, the user should be able to specify how many digits of precision they w[详细]

2023-02-09 22:38 分类：问答
gnuplot alternative with higher time precision

At present I\'m using gnuplot to plot data against a time line. However the precision of the time line is in milliseconds but gnuplot only seems to be able to handle seconds.[详细]

2023-02-06 07:30 分类：问答
jQuery scroll calculations inaccurate at high-speed scrolling

I\'ve run into something very odd. The likelihood that it\'s an error in my code is high, but I can\'t figure out why it\'s happening.[详细]

2023-02-04 08:21 分类：问答
FP precision error on converting XML -> Table & processing it

In SQL Server 2008, I have the value \"0.01\" in an XML attribute. Using OPENXML, I shred the XML into a temp table. If the applicable column is of type real (single precision), it comes out as 0.01 i[详细]

2023-02-01 16:11 分类：问答
Mathematica: "15 digits of Sqrt[x] yield 42+ million digits of x"

I did this in Mathematica to compute Sqrt[5]: a[0] = 2 a[n_] := a[n] = a[n-1] - (a[n-1]^2-5)/2/a[n-1][详细]

2023-01-31 20:26 分类：问答
Determinants of huge matrices in MATLAB

from a simulation problem, I want to calculate complex square matrices on the order of 1000x1000 in MATLAB. Since the values refer to those of Bessel functions, the matrices are not at all sparse.[详细]

2023-01-28 21:10 分类：问答
Double versus float

I have a constant for pi in my code: const float PI = acos(-1); Would it be better to declare it as a double?An answer to another question on this site said floating point operations aren\'t exactly[详细]

2023-01-28 03:38 分类：问答