Large Learning Rate Tames Homogeneity: Convergence and Balancing Effect.
Yuqing WangMinshuo ChenTuo ZhaoMolei TaoPublished in: CoRR (2021)
Keyphrases
- learning rate
- convergence rate
- convergence theorem
- rapid convergence
- convergence speed
- update rule
- adaptive learning rate
- step size
- error function
- conjugate gradient algorithm
- weight update
- learning algorithm
- hidden layer
- multilayer neural networks
- activation function
- global convergence
- faster convergence
- bp neural network algorithm
- natural gradient
- uniform convergence
- multi class
- artificial neural networks
- reinforcement learning