Academy K

Superintelligence Threat Models

HardConceptualsuperintelligencealignmentsafety

Related module: Superintelligence and safety

Problem statement

Analyzing potential risks from advanced AI systems requires precise threat models.

  1. What is the difference between outer and inner alignment?
  2. Describe the mesa-optimization/deceptive alignment concern.
  3. What empirical research could help validate or refute these concerns?

Practise this problem

Verify you understand it—not just that you read the answer.

Learn the foundations

The AI / SI track covers this topic and more in depth.