A Unified Theory of SGD: Variance Reduction, Sampling, Quantization and Coordinate Descent.

Eduard A. Gorbunov Filip Hanzely Peter Richtárik

Published in: AISTATS (2020)

Keyphrases

variance reduction
sample size
monte carlo
gradient estimation
importance sampling
stochastic gradient descent
random sampling
bias variance decomposition
quasi monte carlo
trade off