Academy K

Superposition in Neural Networks

HardConceptualsuperpositioninterpretabilityfeatures

Related module: Alignment fundamentals

Problem statement

Neural networks may represent more features than they have neurons through superposition.

  1. What is the superposition hypothesis?
  2. Explain the toy model that demonstrates superposition.
  3. What are the implications for interpretability?

Practise this problem

Verify you understand it—not just that you read the answer.

Learn the foundations

The AI / SI track covers this topic and more in depth.