Variance Reduction in Policy Gradients
InterviewDerivationpolicy gradientvarianceRL
Related module: RLHF and reward learning
Problem statement
Policy gradient methods suffer from high variance.
- Write the REINFORCE gradient estimator.
- How does the baseline reduce variance without introducing bias?
- Derive why the baseline doesn't introduce bias.