Almost sure convergence rates for Stochastic Gradient Descent and Stochastic Heavy Ball.

Othmane Sebbouh Robert M. Gower Aaron Defazio

Published in: COLT (2021)

Keyphrases

convergence rate
stochastic gradient descent
step size
number of iterations required
convergence speed
learning rate
loss function
least squares
matrix factorization
support vector machine
weight vector
machine learning
learning algorithm
training data
optimal solution
linear programming