Cyclic and Randomized Stepsizes Invoke Heavier Tails in SGD.

Mert Gürbüzbalaban Yuanhan Hu Umut Simsekli Lingjiong Zhu

Published in: CoRR (2023)

Keyphrases

approximate dynamic programming
stochastic gradient descent
long tail
decision forest
machine learning
heavy tails
multi class
online learning
randomized algorithms