Academy K

Score Matching for EBMs

HardDerivationscore matchingenergy-based modelstraining

Related module: Energy-based models

Problem statement

Score matching trains energy-based models without computing the partition function.

The score is sθ(x)=∇xlog⁡pθ(x)=−∇xEθ(x)s_\theta(x) = \nabla_x \log p_\theta(x) = -\nabla_x E_\theta(x).

  1. What is the score matching objective?
  2. Derive the implicit score matching loss that doesn't require knowing ∇xlog⁡pdata\nabla_x \log p_{data}.
  3. What is denoising score matching?

Hints

Practise this problem

Verify you understand it—not just that you read the answer.

Learn the foundations

The Thermodynamic Computing track covers this topic and more in depth.