Measuring the Impact of Gradient Accumulation on Cloud-based Distributed Training.
Zimeng HuangBo JiangTian GuoYunzhuo LiuPublished in: CCGrid (2023)
Keyphrases
- distributed systems
- loosely coupled
- cooperative
- training process
- distributed environment
- training set
- cloud computing
- computing environments
- lightweight
- distributed data
- communication overhead
- fault tolerant
- training examples
- supervised learning
- image processing
- machine learning
- gradient direction
- distributed architecture
- training algorithm
- serious games
- input image
- learning algorithm