Benign Oscillation of Stochastic Gradient Descent with Large Learning Rates.

Miao Lu Beining Wu Xiaodong Yang Difan Zou

Published in: CoRR (2023)

Keyphrases

stochastic gradient descent
learning rate
weight vector
convergence rate
step size
learning algorithm
least squares
loss function
matrix factorization
convergence speed
random forests
online algorithms
lower bound
optical flow
regularization parameter