Reconciling Modern Deep Learning with Traditional Optimization Analyses: The Intrinsic Learning Rate.
Zhiyuan LiKaifeng LyuSanjeev AroraPublished in: CoRR (2020)
Keyphrases
- learning rate
- deep learning
- learning algorithm
- convergence rate
- unsupervised learning
- optimization algorithm
- machine learning
- adaptive learning rate
- convergence speed
- rapid convergence
- delta bar delta
- unsupervised feature learning
- hidden layer
- mental models
- weight vector
- global convergence
- supervised learning
- global optimization
- data mining
- higher order
- perceptron algorithm
- active learning
- convergence theorem
- text mining