Emergent Capabilities in Language Models
InterviewConceptualemergencescalingcapabilities
Related module: Transformers and autoregressive models
Problem statement
Some capabilities appear to "emerge" suddenly as language models scale.
- What is an emergent capability? Give examples.
- What are competing explanations for the emergence phenomenon?
- What are the implications for AI safety?