Academy K

Free Energy and Equilibrium

InterviewDerivationfree energyequilibriumvariational

Related module: Entropy, information and heat

Problem statement

The Helmholtz free energy is F=E−TSF = E - TS where EE is average energy and SS is entropy.

  1. Show that the Boltzmann distribution minimizes free energy.
  2. Write the free energy for a general distribution QQ and show F[Q]≥FeqF[Q] \geq F_{\text{eq}}.
  3. How does this relate to variational inference in machine learning?

Hints

Practise this problem

Verify you understand it—not just that you read the answer.

Learn the foundations

The Thermodynamic Computing track covers this topic and more in depth.