# Inaccurate result in calculation with broken numbers

34

## Problem

Test on the consoles of your browsers:

``````1067.11-1000 = 67.1099999999999
``````

The correct one would be `67.11`

• Could anyone explain this to me?
• And how do I resolve this?
• asked by anonymous 13.02.2014 / 15:23

45

This is because of an inaccuracy in the conversion of the 1067.11 value to the representation of the number in memory ... this imprecision, which is revealed to the subtract the value 1000.

## Inaccuracy factors

Numerical inaccuracies occur through a variety of factors, which stem from the way in which those numbers are represented by the computer.

Floating-point numbers in modern systems are usually represented in accordance with the IEEE 754 natively. Javascript specifically, uses `double` to represent all numbers .

In the case of `double` of the standard IEEE 754 , inaccuracies are caused by two factors:

• Amount of space to represent the number is finite. In this case, this limitation is not what causes inaccuracy.

• Numeric base of the exponent is 2, which can not be aligned with the base of the original number `1067.11` which is in base 10. This is the cause of the imprecision in the current case.

To understand better we have to see how exactly the type `double` is represented.

## Components of a `double`

The `double` of the standard IEEE 754 is formed as follows:

• signal: 1 bit for the signal

• exponent: 11-bit integer, to indicate the value of the displaced exponent 1023 units, or one of two values with special meanings: 0x000 for subnormal values and zero; 0x7FF to represent infinity and NaN. (i.e. 2 x - 1023 , where x is the integer value of the field)

• mantissa: 52 bits, for normal values of the exponent, represents a value ration ranging from 1.0 inclusive to 2.0 exclusive, mathematically `[1, 2[` . For the exponent value 0x000, then it represents subnormal values (i.e. smaller than the lowest representable normal value) or zero; For the value of exponent 0x7FF represents infinity if it is 0, or NaN if it is other than 0.

## Why imprecision appears only after subtracting 1000

Although it does not look like 1067.11 can not be represented exactly by javascript. So how can javascript convert this number? back to string, exactly like "1067.11"?

``````alert(1067.11); // vai mostrar "1067.11", como é possível então?
``````

This is because the javascript implementation is smart ... at the time of convert to string, shows the shorter value that would be representable by this value of `double` .

This happens because a single value of `double` results from several conversions of string for double:

``````1067.1100000000000000000001
1067.11000000000000001
1067.11000000000001
1067.11
1067.10999999999978626875
1067.1099999999997862687453
1067.10999999999978626874509
1067.1099999999998999553285190717
``````

When typing any of the above numbers on the chrome console, the result is `1067.11` ... because this is the shortest string.

But the truth is that you are being deceiving!

• The exact value of the string "1067.11" converted to `double` is `1067.1099999999998999553285190717` .

• Exact value minus 1000 is `67.1099999999998999553285190717` .

• The shortest value representable by the previous double is `67.1099999999999` .

• Conclusion:

``````                          1067.11 - 1000 == 67.1099999999999
é **EXATAMENTE** o mesmo que
1067.1099999999998999553285190717 - 1000 == 67.1099999999998999553285190717
``````

That is:

``````    O que realmente acontece na memória  =>    O que é mostrado para você
1067.1099999999998999553285190717    =>    1067.11
- 1000                                 =>  - 1000
---------------------------------          ------------------
67.1099999999998999553285190717    =>      67.1099999999999
``````

References

My sources of information and learning:

01.04.2014 / 21:27
24

The reason for this is as follows.

The computer does not work well with decimals. As you know, data is represented internally in binary format.

The number 4, for example, is represented by the computer like this:

``````100 (onde 1x2^2 + 0x2^1 + 0x2^0 = 4)
``````

How do you represent a number with decimals? The number is inaccurate because, after the comma, each connected bit represents 2 ^ -n, where n is the house number counted from right to left, from 1. Thus, the number 2.5 would be represented in binary by:

``````2 = 10
0,5 = 0.1
2,5 = 10.1
``````

Now, consider the effort of writing a fraction that is not multiple of 2, like 0.3, using powers of 2:

The first power we would use could be 1/4, which is 0.25. To improve accuracy, we can add 1/32, which is 0.03125. We would then get 0.28125. What if we wanted to increase accuracy even more? It would give to add 1/64, which is 0,015625, and we would get 0.296875 ... Our number is already this way: 0.3 = 0.010011

We could continue to add up with multiple divisors of 2 getting bigger, but we would hardly get to 0.3. This asymptotic behavior is extremely inconvenient and, for a base like 2, extremely frequent.

Thus, we see that the base system 2 is too poor to represent numbers that can not be obtained by fractions of 2. And so this whole imprecision comes. What the computer does is increase (much) the number of bits when accounts involving floats are made. This mitigates the problem but does not solve it. Small programming tips (like multiplying before splitting when possible) also help.

However, it is often not enough. Java (but not JavaScript) provides classes (BigDecimal and BigInteger) for you to handle arbitrary precision numbers. In this case, the calculations are done indirectly, and are much slower compared to the primitive way. But still, they break a gal.

That explains their odd number. It's not a JavaScript bug, it's the way computers work. The same problem will recur in all programming languages.

28.03.2014 / 21:41
12

I think the first part of the question is already well answered ..:)

As for the second, in JavaScript you can format your number in fixed-point using the `Number.prototype.toFixed()`

The method receives a parameter indicating the number of digits:

``````(1067.11-1000).toFixed(2);
``````

In this case, the result will be the string `"67.11"` .

To explicitly work with it again as number:

``````resultado = Number( (1067.11 - 1000).toFixed(2) );
``````

28.03.2014 / 21:19