Academy K

Loss Landscape and Optimization

InterviewConceptualoptimizationloss landscapetraining

Related module: Deep learning fundamentals

Problem statement

The loss landscape of neural networks has complex geometry.

  1. What are saddle points and why are they more common than local minima in high dimensions?
  2. How does batch normalization affect the loss landscape?
  3. What is the "sharp vs flat minima" hypothesis?

Practise this problem

Verify you understand it—not just that you read the answer.

Learn the foundations

The AI / SI track covers this topic and more in depth.