Calculate Distance with Lat / Lon that accounts for Altitude

[dcnick]

Hello. I found Excel formulas and VBA coding to calculate distance between two Lat / Lon coordinates using both the Haversine and Vincenty methods. However, these formulas don't take into account altitude.

Does anyone know of any resources that include altitude in the calculations? I have found a couple of threads that discuss translating to Cartesian coordinates, but those calculations rely on an average earth radius (Great Circle Approach); which affects accuracy. This is the best thread I have been able to find on this discussion.
Google Answers: Using latitude, longitude, and altitude to determine distance

Thanks in advance.

 

[Zack Barresse]

The two best Excel resources I know for calculating distances are:

Latitude And Longitude
http://www.contextures.com/excellatitudelongitude.html

Neither of those incorporate altitude into their calculations. The first link to Chip Pearson's site uses Great Circle Distances equation, whereas the latter link uses Vincenty solution (Vincenty solutions of geodesics on the ellipsoid in JavaScript | Movable Type Scripts). It's an interesting read, the original paper (http://www.ngs.noaa.gov/PUBS_LIB/inverse.pdf), and still holds as the gold standard in distance calculations 40 years after publication. They even programmed the model in FORTRAN! Very impressive indeed.

With this being said, I don't think we could do justice to any calculation here with regard to altitude calculated on a spheroid, assuming the distance travels along such a sphere, with regard to arc and angle. There would be many assumptions, and not enough for any accurate calculation. I don't begin to understand the reasoning behind any of Vincenty's computations. Were we assuming any spherical distance didn't take into account curvature/arc (impossible), then you could simply use the suggestion in the link you posted using the Pythagorean Theorem. As it stands, this doesn't pertain to a spheroid. I understand the OP (original poster) of the link you provided said, "as long as the points along your path are spaced at short intervals," but even still, it's anecdotal in any final computations. It'd basically only be applicable to line-of-sight, in which case any of the above mentioned calculations get fuzzy anyway.

My recommendation would be to either 1) forget about calculating altitude with your values, or 2) use the information in your provided link, along with assuming you're not calculating distance on a spheroid but in a 2D space with no curvature/arc. IMO both cases give inaccuracies too great to calculate.

 

[Macropod]

To account for altitude, simply increase the value for the radius of the earth by the value of the altitude in formulae like:
=ACOS(SIN(RADIANS(Lat1))*SIN(RADIANS(Lat2))+COS(RADIANS(Lat1))*COS(RADIANS(Lat2))*COS(RADIANS(Long1-Long2)))*RadiusEarth
2*ASIN(SQRT((SIN(RADIANS(Lat1-Lat2)/2))^2+COS(RADIANS(Lat1))*COS(RADIANS(Lat2))*(SIN(RADIANS(Long1-Long2)/2))^2))*RadiusEarth
where Lat1, Long1, Lat2 & Long2 are expressed in decimal degrees and RadiusEarth is in mi. or km.
FWIW, the average distance you might use for RadiusEarth is 6367.465km (6356.77km at the poles, 6378.16km at the equator).
On that basis, travelling at altitude you'd simply increase the radius by 1km/1000m asl. Even so, travelling north-south by a given great circle angle is a different distance to travelling east-west around the equator. And, as you can see from the 21.4km difference in distances around the poles than around the equator, even a 10,000m altitude has less effect on the calculated distance travelled than the error inherent in most great circle angle calculations based on the average radius.

When mixing hemispheres, enter Northern and Western coordinates as positive, and Southern and Eastern coordinates as negative.

 

[Zack Barresse]

Not sure I agree here Paul. I think we'd be leading the OP down a dangerous path of assumptions. First, all of the equations references, and used in the formulas, assume the Earth is a perfect spheroid. Second, adding any altitude, no matter if it's calculated from the center of the earth or not, will have an error coefficient based on the distance traveled around the sphere. This isn't accounted for in any of the calculations. At that point why don't we just assume the world is flat and calculate it based on a 2D model? Makes no sense.

 

[Macropod]

adding any altitude, no matter if it's calculated from the center of the earth or not, will have an error coefficient based on the distance traveled around the sphere.

My understanding is that the OP wants the (nominal?) great circle distance travelled at whatever altitude, regardless of whether that's at sea level or 100km above it. Simply increasing the radius by a corresponding amount is all it takes to get that - that much is just basic maths. Yes, the calculations are only approximate, but that's a limitation of the spheroid model, not of the underlying method. Furthermore, I did point out that the calculation is based on the average of the equatorial and polar radii and that you'd get different results with different assumptions. Nevertheless, if someone has a suitable ovoid model, the same principles re changing radii will apply.

A flat-earth model would give a pole-pole distance of about 12714km, whereas the great circle distance is about 19970km (at sea level), making the flat-earth model less than useful.

 

[Zack Barresse]

The point of mentioning a 2D model was that it was less than useful. I understood the request differently, in that they wanted to go beyond the Great Circle Distance calculation, as mentioned in their post.

 

[dcnick]

Thanks, everyone, for the response and discussion. Since the Vincenty method seems to be the more accepted method (aka more accurate) than the Great Circle Distance methods, I was hoping to incorporate the calculations more along those lines. However, the only discussions I have found online to incorporate altitude revolve around the Great Circle Distance method.

If I understand all of my research to date, all methods are just approximations since the earth is not a perfect sphere; and location on the Earth (closer to the poles v the equator) are also a factor. It seems Vincenty tries to account for that by using different Ellipsoid parameters based on location of the Earth. I have set up an Excel spreadsheet to calculate using both Vincenty and a couple of formulas using the Great Circle Distance method, and the differences in results are in the hundredths of feet. That's good enough for my work but might not be good for others. So, in my case, incorporating altitude using the Great Circle Distance method might be good enough.

Cheers,