Academy K

Entropy as Information

Warm-upDerivationentropyinformation theory

Related module: Entropy, information and heat

Problem statement

The Gibbs entropy is defined as:

S=−kB∑iPiln⁡PiS = -k_B \sum_i P_i \ln P_i
  1. Show that for a uniform distribution over NN states, S=kBln⁡NS = k_B \ln N.
  2. What is the entropy of a system in a pure state (one specific microstate)?
  3. Why is entropy maximized for uniform distributions?

Practise this problem

Verify you understand it—not just that you read the answer.

Learn the foundations

The Thermodynamic Computing track covers this topic and more in depth.