Why is parameter averaging beneficial in SGD? An objective smoothing perspective.

Atsushi Nitanda Ryuhei Kikuchi Shugo Maeda Denny Wu

Published in: AISTATS (2024)

Keyphrases

weighted averaging
parameter values
viewpoint
input parameters
curve fitting
smoothing algorithm