Academy K

Emergent Capabilities in Language Models

InterviewConceptualemergencescalingcapabilities

Related module: Transformers and autoregressive models

Problem statement

Some capabilities appear to "emerge" suddenly as language models scale.

  1. What is an emergent capability? Give examples.
  2. What are competing explanations for the emergence phenomenon?
  3. What are the implications for AI safety?

Practise this problem

Verify you understand it—not just that you read the answer.

Learn the foundations

The AI / SI track covers this topic and more in depth.