floating point precision

  1. W

    Accuracy issue (presumably) with low numbers

    My colleague has an issue with summing a list. I've shown the original below, and the permutations I've tried to the right which all result in the correct zero sum. Any idea how this happens? <colgroup><col><col span="4"></colgroup><tbody> Cell Original A>Z Z>A Swapped A1 22.05 -427.85 661...
  2. J

    Excel floating point representation - two 6s are not the same!

    I'm really struggling when trying to understand how Excel deals with number representation. It would appear that (1.2 / 0.2) and (12 / 2) are NOT the same. Although both results appear as '6' in the formula bar, I noticed that in a pivot table, Excel 2013 was not summarising both values...
  3. C

    Precision IEEE sum to 0 error

    I have this spreadsheet with 5 months of data. For reasons unknown to me, 3 months have data that add up to 0 and the remaining 2 months are 0's. When summing the 3 numbers, Excel is smart enough to account for the floating-point issues and it returns 0. However, if you include the two 0's in...
  4. starl

    Floating Decimal Issue? - Rounded values not matching

    I have two cells I compare (via VBA). Both cells are in pivot tables. Both values appear to match, down to the decimal of .295. I've used a simple = formula and Excel says they match. I used format cells to extend the decimal as far as it can and I do not see any extra numbers. Via VBA, I Round...

We've detected that you are using an adblocker.

We have a great community of people providing Excel help here, but the hosting costs are enormous. You can help keep this site running by allowing ads on MrExcel.com.
Allow Ads at MrExcel

Which adblocker are you using?

Disable AdBlock

Follow these easy steps to disable AdBlock

1)Click on the icon in the browser’s toolbar.
2)Click on the icon in the browser’s toolbar.
2)Click on the "Pause on this site" option.
Go back

Disable AdBlock Plus

Follow these easy steps to disable AdBlock Plus

1)Click on the icon in the browser’s toolbar.
2)Click on the toggle to disable it for "mrexcel.com".
Go back

Disable uBlock Origin

Follow these easy steps to disable uBlock Origin

1)Click on the icon in the browser’s toolbar.
2)Click on the "Power" button.
3)Click on the "Refresh" button.
Go back

Disable uBlock

Follow these easy steps to disable uBlock

1)Click on the icon in the browser’s toolbar.
2)Click on the "Power" button.
3)Click on the "Refresh" button.
Go back
Back
Top