Academy K

Hopfield Network as Energy Model

InterviewDerivationHopfield networkassociative memoryenergy

Related module: Ising models and Boltzmann machines

Problem statement

A Hopfield network stores patterns as attractors of an energy landscape.

For neurons si∈{−1,+1}s_i \in \{-1, +1\} and symmetric weights wij=wjiw_{ij} = w_{ji}:

E=−12∑i≠jwijsisjE = -\frac{1}{2}\sum_{i \neq j} w_{ij}s_is_j
  1. Show that asynchronous updates si→sign(∑jwijsj)s_i \to \text{sign}(\sum_j w_{ij}s_j) decrease energy.
  2. How do you store a pattern ξμ\xi^\mu using Hebbian learning?
  3. What is the storage capacity of a Hopfield network?

Practise this problem

Verify you understand it—not just that you read the answer.

Learn the foundations

The Thermodynamic Computing track covers this topic and more in depth.