RLHF Training Pipeline
InterviewConceptualRLHFalignmentreward modeling
Related module: Post-training and alignment methods
Problem statement
Reinforcement Learning from Human Feedback (RLHF) aligns language models to human preferences.
- Describe the three stages of RLHF.
- What is the reward model and how is it trained?
- What is the KL penalty in PPO for RLHF and why is it needed?