The Amelioration of Uncertainty

Free Energy and Statistics

There is a long and sordid history of trying to use thermodynamic analogies in Economics. Everyone senses thermodynamics is relevant somehow, but no one’s able to make the notion specific, true, and useful. The latest effort comes from Andrew Gelman, who speculates on “Free Energy” in Economics. Ironically, “Free Energy” occurs all the time in Gelman’s field of Statistics; a fact which gives clues to its profitable use in Economics.

To begin, consider the exponential family of distributions with some prior equation:

    equation

This family is broad enough to include most of the famous probability distributions. It’s convenient to bring the prior into the exponential:

    equation

Thus finding the most likely value (mode) of the probability distribution is equivalent to minimizing

    equation

This is often done in statistics, although not usually expressed this way. To see the connection between this equation and the Free Energy of physics, consider the Energy as a function of a point in phase space:

(1)   equation

The probability of finding the system in a given micro-state is (from Statistical Mechanics):

    equation

where T is the temperature, k is Boltzmann’s constant, and equation is the partition function. These last two equations can be used to find the probability distribution of the energy. This amounts to a non one-to-one change of variables and requires the “density of energy states” or the volume of phase space equation associated with a particular value of the Energy. After the change of variables one gets:

    equation

The density of states equation is a kind of prior probability. It arises because there are fewer micro-states compatible with lower energies than with higher. As before, this prior can be brought up to the exponent:

    equation

Using Boltzmann’s definition of entropy equation, the exponent F is seen to be the Helmholtz Free Energy:

    equation

From this it’s clear why thermodynamic analogies in Economics fail. An economic equivalent of (1) will likely have a different functional form and not be conserved. This will give any “Economic Free Energy” different qualitative properties from the Helmholtz Free Energy. On a statistical level though the analogy will still hold. So if you want to find the most probable value of some macroeconomic variable, you would still do so by minimizing the “Economic Free Energy”; just as physicists find the most probable Energy by minimizing the Helmholtz Free Energy.

November 15, 2011
1 comment »

    Trackbacks/Pingbacks
    Leave a Reply or trackback