## Free Energy and Statistics

There is a long and sordid history of trying to use thermodynamic analogies in Economics. Everyone senses thermodynamics is relevant somehow, but no one’s able to make the notion specific, true, and useful. The latest effort comes from Andrew Gelman, who speculates on “Free Energy” in Economics. Ironically, “Free Energy” occurs all the time in Gelman’s field of Statistics; a fact which gives clues to its profitable use in Economics.

To begin, consider the exponential family of distributions with some prior :

This family is broad enough to include most of the famous probability distributions. It’s convenient to bring the prior into the exponential:

Thus finding the most likely value (mode) of the probability distribution is equivalent to minimizing

This is often done in statistics, although not usually expressed this way. To see the connection between this and the Free Energy of physics, consider the Energy as a function of a point in phase space:

(1)

The probability of finding the system in a given micro-state is (from Statistical Mechanics):

where T is the temperature, k is Boltzmann’s constant, and is the partition function. These last two equations can be used to find the probability distribution of the energy. This amounts to a non one-to-one change of variables and requires the “density of energy states” or the volume of phase space associated with a particular value of the Energy. After the change of variables one gets:

The density of states is a kind of prior probability. It arises because there are fewer micro-states compatible with lower energies than with higher. As before, this prior can be brought up to the exponent:

Using Boltzmann’s definition of entropy , the exponent F is seen to be the Helmholtz Free Energy:

From this it’s clear why thermodynamic analogies in Economics fail. An economic equivalent of (1) will likely have a different functional form and not be conserved. This will give any “Economic Free Energy” different qualitative properties from the Helmholtz Free Energy. On a statistical level though the analogy will still hold. So if you want to find the most probable value of some macroeconomic variable, you would still do so by minimizing the “Economic Free Energy”; just as physicists find the most probable Energy by minimizing the Helmholtz Free Energy.