1-bit LAMB: Communication Efficient Large-Scale Large-Batch Training with LAMB's Convergence Speed.
Conglong LiAmmar Ahmad AwanHanlin TangSamyam RajbhandariYuxiong HePublished in: HIPC (2022)
Keyphrases
- convergence speed
- convergence rate
- particle swarm optimization
- differential evolution
- particle swarm optimization algorithm
- global convergence
- search capabilities
- step size
- learning rate
- global search
- pso algorithm
- training speed
- faster convergence
- training set
- particle swarm optimization pso
- bp algorithm
- artificial neural networks
- population diversity
- learning algorithm