Academy K

Neural Scaling Laws

InterviewDerivationscalingpower lawscompute

Related module: Deep learning fundamentals

Problem statement

Neural network performance follows predictable scaling laws.

  1. State the Chinchilla scaling law relating optimal model size to compute budget.
  2. If you have 10×10\times more compute, how should you allocate it optimally?
  3. What are the limitations of scaling law extrapolations?

Practise this problem

Verify you understand it—not just that you read the answer.

Learn the foundations

The AI / SI track covers this topic and more in depth.