Provable Super-Convergence With a Large Cyclical Learning Rate.
Samet OymakPublished in: IEEE Signal Process. Lett. (2021)
Keyphrases
- learning rate
- convergence rate
- convergence theorem
- rapid convergence
- convergence speed
- update rule
- adaptive learning rate
- weight update
- error function
- step size
- hidden layer
- global convergence
- learning algorithm
- conjugate gradient algorithm
- multilayer neural networks
- weight vector
- faster convergence
- particle swarm optimization
- neural network
- activation function
- training algorithm
- global optimization
- natural gradient
- differential evolution
- optimization algorithm