Simple subtraction not giving correct solution

JoshSnitz

New Member
Joined
Aug 29, 2017
Messages
6
Hi all,

I'm having a strange problem. I have a formula subtracting one cell's value from another - the resulting number is incorrect by a very small decimal amount (like 1*10^-14). The initial two numbers were typed in so they should be exact values. Anyone know what is going on? Screenshot is below.


DEKnL
XNSA2Vh.png
 

Excel Facts

Does the VLOOKUP table have to be sorted?
No! when you are using an exact match, the VLOOKUP table can be in any order. Best-selling items at the top is actually the best.
The short answer.... This is a common anomaly of 64-bit binary floating-point, which is how Excel represents numeric values internally.

The work-around.... Whenever you expect a calculation that involves numbers with decimal fractions to be accurate to n decimal places, explicitly round to that number of decimal places. Do not round to an arbitrary number of decimal places (like 10), as some people suggest.

For example, change the formula to =ROUND(R6-Q6,2).

I round to 2 decimal places instead of just 1 because some other numbers that we see in the column have 2 decimal places. So I presume you expect results to be accurate to 2 decimal places.

The long answer.... Numeric values are represented as the sum of 53 consecutive powers of 2 ("bits") times an exponential factor. Consequently, most decimal fractions cannot be represented exactly. And most binary calculations result in infinitesimal differences with the decimal calculation that we might do manually.

Moreover, because some of the bits must be used to represent the integer part of a number, there might be fewer bits (a short binary sum) to represent the fractional part. Consequently, the same decimal fraction might have different approximations in different numbers, depending on the magnitude of the integer part. That can result in additional infinitesimal differences with manual decimal calculations.

For example, IF(10.01 - 10 = 0.01, TRUE) returns FALSE(!) because the approximation of x.01 when x=10 is different from the approximation of x.01 when x=0.
 
Last edited:
Upvote 0
Thanks for the replys! I agree that the issue must be from the way the data is stored. I was inclined to use the round solution but also wanted to find out what the cause was, cheers!
 
Upvote 0
Hi Guys,

Is below issue also a Floating Point Error? It seems such a simple calculation, right?

I am using Windows 11 with Office 365.

Looking forward to your reaction. Thanks in advance.

Best regards,
Bas
The Netherlands


issue with subtraction.jpg
 
Upvote 0

Forum statistics

Threads
1,223,954
Messages
6,175,601
Members
452,658
Latest member
GStorm

We've detected that you are using an adblocker.

We have a great community of people providing Excel help here, but the hosting costs are enormous. You can help keep this site running by allowing ads on MrExcel.com.
Allow Ads at MrExcel

Which adblocker are you using?

Disable AdBlock

Follow these easy steps to disable AdBlock

1)Click on the icon in the browser’s toolbar.
2)Click on the icon in the browser’s toolbar.
2)Click on the "Pause on this site" option.
Go back

Disable AdBlock Plus

Follow these easy steps to disable AdBlock Plus

1)Click on the icon in the browser’s toolbar.
2)Click on the toggle to disable it for "mrexcel.com".
Go back

Disable uBlock Origin

Follow these easy steps to disable uBlock Origin

1)Click on the icon in the browser’s toolbar.
2)Click on the "Power" button.
3)Click on the "Refresh" button.
Go back

Disable uBlock

Follow these easy steps to disable uBlock

1)Click on the icon in the browser’s toolbar.
2)Click on the "Power" button.
3)Click on the "Refresh" button.
Go back
Back
Top