Academy K

Capability Elicitation and Sandbagging

InterviewConceptualcapabilitiesevaluationdeception

Related module: Alignment fundamentals

Problem statement

Capability elicitation is the problem of determining what a model can actually do.

  1. Why might standard evaluations underestimate model capabilities?
  2. What is "sandbagging" and why is it concerning?
  3. How can evaluators improve capability elicitation?

Practise this problem

Verify you understand it—not just that you read the answer.

Learn the foundations

The AI / SI track covers this topic and more in depth.