Stochastic Collapse: How Gradient Noise Attracts SGD Dynamics Towards Simpler Subnetworks.

Feng Chen Daniel Kunin Atsushi Yamamura Surya Ganguli

Published in: NeurIPS (2023)

Keyphrases

gradient estimation
noise reduction
noise level
monte carlo
dynamical systems
image noise
additive noise
random noise
missing data
signal to noise ratio
gradient information
gradient direction
cross validation
network structure
dynamic model
image structure