Eigencurve: Optimal Learning Rate Schedule for SGD on Quadratic Objectives with Skewed Hessian Spectrums.

Rui Pan Haishan Ye Tong Zhang

Published in: ICLR (2022)

Keyphrases

learning rate
convergence rate
learning algorithm
error function
hidden layer

convergence speed
multilayer neural networks
scheduling problem
step size
training speed

weight vector
rapid convergence
dynamic programming
adaptive learning rate
genetic programming

least squares
optimal solution
convergence theorem
machine learning