Academy K

RLHF Training Pipeline

InterviewConceptualRLHFalignmentreward modeling

Related module: Post-training and alignment methods

Problem statement

Reinforcement Learning from Human Feedback (RLHF) aligns language models to human preferences.

  1. Describe the three stages of RLHF.
  2. What is the reward model and how is it trained?
  3. What is the KL penalty in PPO for RLHF and why is it needed?

Practise this problem

Verify you understand it—not just that you read the answer.

Learn the foundations

The AI / SI track covers this topic and more in depth.