Activation Patching for Circuit Analysis
HardDerivationactivation patchingcircuitsinterpretability
Related module: Alignment fundamentals
Problem statement
Activation patching is a technique for understanding which components contribute to model behavior.
- Describe the activation patching methodology.
- What is the difference between patching activations and patching attention patterns?
- Design an experiment using activation patching to find which attention heads are responsible for copying.