Loss Landscape and Optimization
InterviewConceptualoptimizationloss landscapetraining
Related module: Deep learning fundamentals
Problem statement
The loss landscape of neural networks has complex geometry.
- What are saddle points and why are they more common than local minima in high dimensions?
- How does batch normalization affect the loss landscape?
- What is the "sharp vs flat minima" hypothesis?