Implicit Bias of the Step Size in Linear Diagonal Neural Networks.
Mor Shpigel NacsonKavya RavichandranNathan SrebroDaniel SoudryPublished in: ICML (2022)
Keyphrases
- step size
- neural network
- steepest descent method
- convergence rate
- convergence speed
- cost function
- evolutionary programming
- back propagation
- pattern recognition
- least mean square
- adaptive filter
- faster convergence
- fuzzy logic
- artificial neural networks
- gradient method
- hessian matrix
- line search
- stochastic gradient descent
- temporal difference
- learning rate
- covariance matrix
- high quality