In aviation, altitude has always been determined from a static pressure measurement at the aircraft and converted to altitude by a formula that uses a so called Standard Atmosphere. Even though we can now determine our altitude far more accurately from a WAAS GPS position solution, barometric altitude is still the international standard so it is important to understand a little of the math that converts pressure to altitude at your location, and to know why there are errors and how big they might be.

There is a simple relationship between a very small change in pressure (dp) at your location caused by a very small change in elevation (dz) known as the Barometric Law. The Law is derived mathematically from the fact that the air pressure at any height in a column of air with unit area is just the weight of all the air above it. Before writing down the equation relating pressure and height, a scale factor H for height can be introduced. The product mgH = kT defines it, where T is the temperature in degrees Kelvin, g = 9.8 m/sec/sec is the acceleration due to gravity, k is Boltzmann's constant, and m is the average mass of the air molecules in a column of air. That mass is about 29 times the mass of a proton, determined by the mix of oxygen and nitrogen and other trace gasses in the air.

You can compute H at a particular altitude if you know the temperature at that altitude. At sea level on a standard 15° C day the temperature is T = 288° Kelvin, (add 273° to the temperature in Centigrade). Then, you get H = 27,500 feet. Since H is proportional to temperature, it will typically be less than this as you go up in altitude (unless there is a temperature inversion). So, at different temperatures change the value of H in the ratio of T/288 to 27,500. At 5,000 ft, with a typical temperature drop of 10° C you get H = 26,545.

With this definition of H, the barometric law is H dp = - p dz, which is an equation relating the differential changes dp and dz. So, if we make a pressure decrease of dp = - 0.01 in Hg, then the change in height is dz = H/100p At sea level, with p = 29.92 in Hg and H = 27,500 ft, dz = 9.19 ft. For a change of 0.1 in Hg, the altitude is about 91.9 ft. (Remember the old rule that a 100 ft altitude change is about 0.1 in Hg).

Writing the barometric law as a derivative you get dp/dz = -p/H. Calculus students will recognize the solution as an exponential when H is constant (which it is not). That solution is p(z) = p(0) exp{-A} with A, defined as z/H, being a normalized height (scaled by the factor H). Here, p(z) means the pressure as a function of altitude z, and p(0) means the pressure where z = 0. You could choose z = 0 at mean seal level, or at a reporting station at an altitude above sea level. But, altimeter settings are not reported as the actual pressure there, but instead as a correction to the sea level pressure at that location. So if you want to take z = 0 at the reporting station you would first have to calculate the pressue at the station elevation using the Standard Atmosphere Model and the baro correction there to get the answer. Then, you could use that as the starting pressure p(0) and calculate pressures AGL as z increases.

The barometric law shows that the pressure drops exponentially with altitude. This would be exactly true if there were not a variation of H with altitude. That is, H(z) is not a constant as z increases. But at 10,000 ft, with a typical drop in temperature of 2° C per 1000 ft, the temperature would be 268 ° K and not 288 ° K. That's about a 7% error, so estimates using H = constant are approximately correct. In a minute I'll give you an equation for pressure (p) versus altitude (z) that uses the Standard model to represent H(z) as it changes with altitude. But for now, let's just assume H is constant and invert the solution (take the natural logarithm of each side) to get

z = H L(z) with L(z) = ln {p(0)/p(z)}

Let's illustrate this by asking what altitude goes with a measured pressure at your aircraft, where z = h and p(h) = 21.43 (inches of Mercury). We'll take the sea level pressure to be p(0) = 30.13. The pressure ratio is 1.405 and the natural log of that is L(h) = 0.34. So, your altitude is z = h =LH or 9,350 ft. We used the specific value of L when z = h.

The problem in calculating altitude from your local pressure is that you have to know the temperature of each altitude slice below you, specifically for each parcel of air between you and the ground station that's reporting a corrected barometric pressure (referenced to MSL at the station). If you know the variation of temperature with height then you know how H(z) changes in direct proportion. With that, you can integrate the barometric equation and then (again) take the natural logarithm of each side to find the factor A(z) as a function of z; The new definition of A(z) when H is not constant is;

A(z) = ∫ dz/H = L(z) = ln {p(0)/p(z)}

The integration continues to a height at which A matches the log of the pressure ratio (sea level pressure divided by the pressure at your altitude). This is called an integral equation, and could be solved if you knew the temperature profile, and thereby the function H(z). For those not familiar with integrals I'll tell you how to calculate A(z). Integration is just the addition of very small pieces, so in a column of air starting at sea level, make very small vertical slices (thickness dz) of the column. Compute dz/H and move to the next slice, with the same thickness dz but a slightly lower H. Add all these together, all the way up to an altitude z and that equals A(z). To find your altitude, z = h, you must find where A(h) matches the logarithmic pressure ratio L(h).

If you launch a balloon with instruments that record
temperatures all the way to your altitude you could do the integration of dz/H.
Since this is impractical, a Standard Atmosphere model is used to characterize
the temperature change with altitude and to define sea level standard
conditions. The model is that
the sea level temperature is 288° K (15 °C), the pressure is 29.92 in Hg, and the temperature drops at an
average lapse rate of 1.98 °C per thousand feet.
Corrections to non standard MSL pressure are taken care of by setting the local
baro correction in the Kollsman window.
Mathematically, the model temperature is T(z) =
Tmsl (1
- 2a z), where the factor a is defined from2a
= 1.98/(1000T_{msl}). Using
this function T(z) we can do the integration of dz/H. The integration can be done
exactly, but the form of the answer is messy, and a good approximate answer (for ah
<< 1) is more illuminating. Using this approximation in the integration is good
up to fairly high altitudes, and results in the following equation for h. (We
want to find when z = h --- we use z for the variable altitude in the integration, and
h for our fixed altitude.)

L(h) = ln{p(0)/p(h)} = A(h) ≈ h(1 + ah)/H

This is a quadratic formula for h as a function of the logrithmic pressure ratio L(h) at our altitude. It too could be solved exactly, but an approximate answer consistent with the approximation ah << 1 is

h ≈ LH(1 - aLH)

Again, we are using L = L(h), the value for L when z = h. In our last example the logarithmic ratio at z = h was L(h) = 0.34 and the answer was h = LH = 9,350 ft. But here, the quantity aLH is 0.032 (a 3.2% correction) so the altitude is 0.968(9,350) = 9,051 ft.

In the example problem we showed that there was a 3.2% change to our altitude calculation when temperature is not constant, but instead drops with altitude according to the Standard Atmosphere model. This is a model correction, not an error. Altimeters are constructed to display altitude according to this model. So our correction simply accounts for the variation of H in the Standard model. But if we found a day when the temperature was in fact (approximately) constant for the first 9,000 ft above MSL, then that correction would indeed represent the error from what our altimeter shows. Our altitude would be 9,350 ft but the altimeter would show 9,051 ft.

You could now use this formula to compute errors from simple deviations from the Standard Model, like different lapse rates and different MSL temperatures. You could for example verify the old admonishment, from high to low, look out below. But you couldn't compute the actual error without complete data on the atmosphere. For example, suppose the lapse rate on the day we're flying is 3° C per thousand. The correction from the model is proportional to the lapse rate. So a 50% increase in lapse rate means a 50%, change in the model correction, from 300 ft to 450 ft in this example. In that case, the error in our altitude (according to these assumptions) is 150 ft. We would be 150 ft below the model estimate of 9,051 ft.