Perplexity as Evaluation Metric
Warm-upDerivationevaluationperplexitylanguage models
Related module: Evaluation and interpretability
Problem statement
Perplexity is a standard metric for language models.
- Define perplexity mathematically.
- If a model achieves perplexity 10 on a test set, what does this mean intuitively?
- What are the limitations of perplexity as an evaluation metric?