Achieving small-batch accuracy with large-batch scalability via Hessian-aware learning rate adjustment.
Sunwoo LeeChaoyang HeSalman AvestimehrPublished in: Neural Networks (2023)
Keyphrases
- learning rate
- convergence rate
- high accuracy
- training speed
- error function
- hidden layer
- convergence speed
- learning algorithm
- step size
- neural network
- training algorithm
- online algorithms
- rapid convergence
- multilayer neural networks
- particle swarm optimization
- small number
- evolutionary algorithm
- hessian matrix
- adaptive learning rate