Mode Connectivity in Loss Landscapes
HardConceptualloss landscapemode connectivityoptimization
Related module: Deep learning fundamentals
Problem statement
Mode connectivity refers to the structure connecting different minima in the loss landscape.
- What is mode connectivity and what does it imply about loss landscapes?
- Distinguish between linear mode connectivity and path connectivity.
- How is mode connectivity related to neural network trainability?