1-bit LAMB: Communication Efficient Large-Scale Large-Batch Training with LAMB's Convergence Speed.
Conglong LiAmmar Ahmad AwanHanlin TangSamyam RajbhandariYuxiong HePublished in: CoRR (2021)
Keyphrases
- convergence speed
- particle swarm optimization algorithm
- differential evolution
- particle swarm optimization
- convergence rate
- step size
- global convergence
- training speed
- learning rate
- global search
- bp neural network algorithm
- faster convergence
- pso algorithm
- training set
- firefly algorithm
- batch mode
- ant colony optimization algorithm
- improve the convergence speed
- neural network