Gradient Descent on Neural Networks Typically Occurs at the Edge of Stability.
Jeremy M. CohenSimran KaurYuanzhi LiJ. Zico KolterAmeet TalwalkarPublished in: ICLR (2021)
Keyphrases
- neural network
- learning rules
- cost function
- pattern recognition
- neural network model
- neural nets
- artificial neural networks
- back propagation
- genetic algorithm
- objective function
- edge detection
- lyapunov function
- loss function
- stability analysis
- edge information
- multi layer
- fuzzy systems
- multilayer perceptron
- weighted graph
- lyapunov theory
- expert systems
- rule extraction
- fuzzy neural network
- network architecture
- learning algorithm
- recurrent neural networks
- feed forward
- data sets
- self organizing maps
- sufficient conditions
- decision trees