The large learning rate phase of deep learning: the catapult mechanism.
Aitor LewkowyczYasaman BahriEthan DyerJascha Sohl-DicksteinGuy Gur-AriPublished in: CoRR (2020)
Keyphrases
- learning rate
- deep learning
- convergence rate
- learning algorithm
- machine learning
- unsupervised learning
- unsupervised feature learning
- adaptive learning rate
- hidden layer
- rapid convergence
- convergence theorem
- convergence speed
- mental models
- weakly supervised
- delta bar delta
- higher order
- denoising
- support vector
- object recognition
- deep architectures
- viewpoint
- feature space