precision

  1. U

    Date edit VBA

    Any help on this would be greatly appreciated. I have already tried recording a macro but to no avail. I have a wkbk with 6 sheets, and each one has a date corresponding to the data in the form of 2019:129:14:18:31.274 YYYY:DDD(day of the year as in 129th day):HH:MM:SS.sss I would like to turn...
  2. H

    Rounding and Precision Errors

    Hello, In Excel/VBA, I have variables: Q and R. They are both a function of x (complicated formula - but no need to describe it here). D = Q^3 + R^2 So I have: x in column A Q in column B R in column C D in column D At one point, Q^3 so small that D = R^2. I am adding a small number (Q^3)...
  3. L

    Display seconds always "00"

    Hi, TEXT(A1,"hh:mm:ss##:##:00") is not working. How can I format the time as it will show seconds always "00"? I don't want to use any floor or round function because the problem I face is already because of precision overflow. So I want to take a shortcut with text formatting. Eg: 01:00:01 =>...
  4. D

    Not sure if this is a V-Lookup, Index, or what!

    Trying to provide a Quote sheet that lists the Materials,Grade of Materials, Raw Shape and Finish for a project. The Material selection can easily bedone through a Drop Down data validation list, but based on that selection I want the nextcell to only offer the Grades of Material associated to...
  5. A

    Vba Macro and Excel Sheet results are not the same

    Hello! I am doing an excel sheet with a very big table for some calculations (tens of thousands of rows and many columns too). I must deliver the excel file with quite nice appearance, so I decided to make a VBA Macro. Each row takes into account the previous one, and I have found that an...
  6. P

    Function returns value to cell as Single Function is defined to return Double

    Function Haversine has correct value in debugger but in cell it has the same value as Haversine2. Is this a known bug? Public Function Haversine(lat1 As Double, long1 As Double, lat2 As Double, long2 As Double) As Double Dim temp As Double Dim temp1 As Double Dim temp2 As Double Dim temp3 As...
  7. M

    Chart DataTable Format Mystery

    [SOLVED: workaround found, see bottom] (Before you reply, I do understand that most decimal numbers can not be expressed exactly in binary format, even simple numbers such as .3. I'm trying to limit the number of digits after the decimal that are shown.) I'm trying to cleanup the data table...
  8. A

    Excel Calculation Issue

    I have an interesting issue I just found in one of my Excel workbooks, I was hoping one of the brilliant minds on Mr. Excel could shed some light on this. I have three values in A1, A2 and A3. All three of these values were derived from formulas in another worksheet, but I pasted the values...
  9. J

    Problem with precision while adding parentheses

    Hi Guys I'm performing really basics calculations in excel 2007, however, when I add parentheses at the beginning and end of a formula (which mathematically shouldn't change a thing), I get a precision error. Here is what I mean with: =C29-(C8-C14-((C8-C14-C19)/2)) I get...
  10. Jerry Sullivan

    Precision Limitations Using SUMIF vs SUM

    I wasn't aware that precision could be an issue using SUMIF with dollars and cents values (2 decimals), but apparently it is. In the example below, both nearly identical Accounts have a SUM of exactly zero, however the list of Account B yields a SUMIF result slightly less than zero.Excel...

We've detected that you are using an adblocker.

We have a great community of people providing Excel help here, but the hosting costs are enormous. You can help keep this site running by allowing ads on MrExcel.com.
Allow Ads at MrExcel

Which adblocker are you using?

Disable AdBlock

Follow these easy steps to disable AdBlock

1)Click on the icon in the browser’s toolbar.
2)Click on the icon in the browser’s toolbar.
2)Click on the "Pause on this site" option.
Go back

Disable AdBlock Plus

Follow these easy steps to disable AdBlock Plus

1)Click on the icon in the browser’s toolbar.
2)Click on the toggle to disable it for "mrexcel.com".
Go back

Disable uBlock Origin

Follow these easy steps to disable uBlock Origin

1)Click on the icon in the browser’s toolbar.
2)Click on the "Power" button.
3)Click on the "Refresh" button.
Go back

Disable uBlock

Follow these easy steps to disable uBlock

1)Click on the icon in the browser’s toolbar.
2)Click on the "Power" button.
3)Click on the "Refresh" button.
Go back
Back
Top