Academy K

Learning in Boltzmann Machines

HardDerivationBoltzmann machinelearning rulegradient

Related module: Ising models and Boltzmann machines

Problem statement

A Boltzmann machine with visible units vv and hidden units hh has energy:

E(v,h)=−∑i<jwijsisj−∑ibisiE(v, h) = -\sum_{i<j}w_{ij}s_is_j - \sum_i b_is_i

where s=(v,h)s = (v, h) are all units.

  1. Derive the gradient of the log-likelihood with respect to wijw_{ij}.
  2. What are the "positive" and "negative" phases of learning?
  3. Why is training general Boltzmann machines hard?

Practise this problem

Verify you understand it—not just that you read the answer.

Learn the foundations

The Thermodynamic Computing track covers this topic and more in depth.