Adaptive scaling of the learning rate by second order automatic differentiation.
Frédéric de GournayAlban GossardPublished in: CoRR (2022)
Keyphrases
- learning rate
- convergence rate
- learning algorithm
- rapid convergence
- convergence speed
- error function
- adaptive learning rate
- hidden layer
- weight vector
- multilayer neural networks
- training algorithm
- activation function
- delta bar delta
- natural gradient
- control parameters
- genetic programming
- bp neural network algorithm
- optimal solution