Cross-Entropy Loss Derivation
Warm-upDerivationloss functionscross-entropyclassification
Related module: Learning from data and backpropagation
Problem statement
The cross-entropy loss for classification is:
where is the one-hot target and is the predicted probability distribution.
- For a single correct class , simplify this expression.
- Show that minimizing cross-entropy is equivalent to maximizing log-likelihood.
- What is the gradient where ?