Superposition in Neural Networks
HardConceptualsuperpositioninterpretabilityfeatures
Related module: Alignment fundamentals
Problem statement
Neural networks may represent more features than they have neurons through superposition.
- What is the superposition hypothesis?
- Explain the toy model that demonstrates superposition.
- What are the implications for interpretability?