Reward Hacking and Specification Gaming
InterviewConceptualreward hackingalignmentspecification
Related module: Alignment fundamentals
Problem statement
Reward hacking occurs when an agent optimizes a proxy reward in ways that violate the designer's intent.
- Give three examples of reward hacking in RL systems.
- Why is reward hacking sometimes called "Goodhart's Law for AI"?
- What approaches can mitigate reward hacking?