Inaccurate result in calculation with broken numbers



Test on the consoles of your browsers:

1067.11-1000 = 67.1099999999999

The correct one would be 67.11

  • Could anyone explain this to me?
  • And how do I resolve this?
  • asked by anonymous 13.02.2014 / 15:23

    3 answers


    Short answer

    This is because of an inaccuracy in the conversion of the 1067.11 value to the representation of the number in memory ... this imprecision, which is revealed to the subtract the value 1000.

    Long answer

    Inaccuracy factors

    Numerical inaccuracies occur through a variety of factors, which stem from the way in which those numbers are represented by the computer.

    Floating-point numbers in modern systems are usually represented in accordance with the IEEE 754 natively. Javascript specifically, uses double to represent all numbers .

    In the case of double of the standard IEEE 754 , inaccuracies are caused by two factors:

    • Amount of space to represent the number is finite. In this case, this limitation is not what causes inaccuracy.

    • Numeric base of the exponent is 2, which can not be aligned with the base of the original number 1067.11 which is in base 10. This is the cause of the imprecision in the current case.

    To understand better we have to see how exactly the type double is represented.

    Components of a double

    The double of the standard IEEE 754 is formed as follows:

    • signal: 1 bit for the signal

    • exponent: 11-bit integer, to indicate the value of the displaced exponent 1023 units, or one of two values with special meanings: 0x000 for subnormal values and zero; 0x7FF to represent infinity and NaN. (i.e. 2 x - 1023 , where x is the integer value of the field)

    • mantissa: 52 bits, for normal values of the exponent, represents a value ration ranging from 1.0 inclusive to 2.0 exclusive, mathematically [1, 2[ . For the exponent value 0x000, then it represents subnormal values (i.e. smaller than the lowest representable normal value) or zero; For the value of exponent 0x7FF represents infinity if it is 0, or NaN if it is other than 0.

    Why imprecision appears only after subtracting 1000

    Although it does not look like 1067.11 can not be represented exactly by javascript. So how can javascript convert this number? back to string, exactly like "1067.11"?

    alert(1067.11); // vai mostrar "1067.11", como é possível então?

    This is because the javascript implementation is smart ... at the time of convert to string, shows the shorter value that would be representable by this value of double .

    This happens because a single value of double results from several conversions of string for double:


    When typing any of the above numbers on the chrome console, the result is 1067.11 ... because this is the shortest string.

    But the truth is that you are being deceiving!

    • The exact value of the string "1067.11" converted to double is 1067.1099999999998999553285190717 .

    • Exact value minus 1000 is 67.1099999999998999553285190717 .

    • The shortest value representable by the previous double is 67.1099999999999 .

    • Conclusion:

                                1067.11 - 1000 == 67.1099999999999
                                  é **EXATAMENTE** o mesmo que
      1067.1099999999998999553285190717 - 1000 == 67.1099999999998999553285190717  

    That is:

        O que realmente acontece na memória  =>    O que é mostrado para você
        1067.1099999999998999553285190717    =>    1067.11
      - 1000                                 =>  - 1000
        ---------------------------------          ------------------
          67.1099999999998999553285190717    =>      67.1099999999999


    My sources of information and learning:

    01.04.2014 / 21:27

    Source:    link

    The reason for this is as follows.

    The computer does not work well with decimals. As you know, data is represented internally in binary format.

    The number 4, for example, is represented by the computer like this:

    100 (onde 1x2^2 + 0x2^1 + 0x2^0 = 4)

    How do you represent a number with decimals? The number is inaccurate because, after the comma, each connected bit represents 2 ^ -n, where n is the house number counted from right to left, from 1. Thus, the number 2.5 would be represented in binary by:

    2 = 10 
    0,5 = 0.1
    2,5 = 10.1

    Now, consider the effort of writing a fraction that is not multiple of 2, like 0.3, using powers of 2:

    The first power we would use could be 1/4, which is 0.25. To improve accuracy, we can add 1/32, which is 0.03125. We would then get 0.28125. What if we wanted to increase accuracy even more? It would give to add 1/64, which is 0,015625, and we would get 0.296875 ... Our number is already this way: 0.3 = 0.010011

    We could continue to add up with multiple divisors of 2 getting bigger, but we would hardly get to 0.3. This asymptotic behavior is extremely inconvenient and, for a base like 2, extremely frequent.

    Thus, we see that the base system 2 is too poor to represent numbers that can not be obtained by fractions of 2. And so this whole imprecision comes. What the computer does is increase (much) the number of bits when accounts involving floats are made. This mitigates the problem but does not solve it. Small programming tips (like multiplying before splitting when possible) also help.

    However, it is often not enough. Java (but not JavaScript) provides classes (BigDecimal and BigInteger) for you to handle arbitrary precision numbers. In this case, the calculations are done indirectly, and are much slower compared to the primitive way. But still, they break a gal.

    That explains their odd number. It's not a JavaScript bug, it's the way computers work. The same problem will recur in all programming languages.

    28.03.2014 / 21:41

    I think the first part of the question is already well answered ..:)

    As for the second, in JavaScript you can format your number in fixed-point using the Number.prototype.toFixed()

    The method receives a parameter indicating the number of digits:


    In this case, the result will be the string "67.11" .

    To explicitly work with it again as number:

    resultado = Number( (1067.11 - 1000).toFixed(2) );
    28.03.2014 / 21:19