Implicit Regularization and Convergence for Weight Normalization.
Xiaoxia WuEdgar DobribanTongzheng RenShanshan WuZhiyuan LiSuriya GunasekarRachel WardQiang LiuPublished in: NeurIPS (2020)
Keyphrases
- weight update
- convergence proof
- normalization method
- convergence speed
- parameter selection
- weight assignment
- preprocessing
- regularization parameter
- neural network
- regularization methods
- regularization method
- faster convergence
- initial conditions
- iterative algorithms
- weighting scheme
- global convergence
- convergence analysis
- convergence rate
- stochastic approximation
- weight function
- image restoration
- maximum likelihood
- artificial neural networks
- conjugate gradient algorithm
- smoothing parameter
- feature selection
- data sets
- projection operator