Reconciling Modern Deep Learning with Traditional Optimization Analyses: The Intrinsic Learning Rate.
Zhiyuan LiKaifeng LyuSanjeev AroraPublished in: NeurIPS (2020)
Keyphrases
- learning rate
- deep learning
- learning algorithm
- convergence rate
- machine learning
- unsupervised learning
- unsupervised feature learning
- hidden layer
- rapid convergence
- optimization algorithm
- convergence speed
- weakly supervised
- adaptive learning rate
- global optimization
- convergence theorem
- co occurrence
- objective function
- mental models
- weight vector