Academy K

Contrastive Divergence Training

InterviewDerivationcontrastive divergencetrainingRBM

Related module: Energy-based models

Problem statement

Contrastive Divergence (CD) is an approximate algorithm for training energy-based models.

  1. Derive the gradient of the log-likelihood for an EBM.
  2. Why is this gradient intractable?
  3. How does CD-k approximate this gradient?

Practise this problem

Verify you understand it—not just that you read the answer.

Learn the foundations

The Thermodynamic Computing track covers this topic and more in depth.