Academy K

Linear Probing for Interpretability

InterviewConceptualinterpretabilityprobingrepresentations

Related module: Evaluation and interpretability

Problem statement

Linear probes are used to understand what information is encoded in neural network representations.

  1. What is a linear probe and how is it trained?
  2. If a linear probe achieves high accuracy, what can and can't we conclude?
  3. Design a probing experiment to test if a language model encodes syntactic structure.

Practise this problem

Verify you understand it—not just that you read the answer.

Learn the foundations

The AI / SI track covers this topic and more in depth.