Eigencurve: Optimal Learning Rate Schedule for SGD on Quadratic Objectives with Skewed Hessian Spectrums.
Rui PanHaishan YeTong ZhangPublished in: ICLR (2022)
Keyphrases
- learning rate
- convergence rate
- learning algorithm
- error function
- hidden layer
- convergence speed
- multilayer neural networks
- scheduling problem
- step size
- training speed
- weight vector
- rapid convergence
- dynamic programming
- adaptive learning rate
- genetic programming
- least squares
- optimal solution
- convergence theorem
- machine learning