CO2: Efficient Distributed Training with Full Communication-Computation Overlap.
Weigao SunZhen QinWeixuan SunShidi LiDong LiXuyang ShenYu QiaoYiran ZhongPublished in: CoRR (2024)
Keyphrases
- communication overhead
- efficient computation
- lightweight
- communication cost
- cooperative
- distributed systems
- spatially distributed
- mobile agents
- cost effective
- test set
- open systems
- multimedia communication
- distributed computation
- training set
- computer networks
- online learning
- multi party
- artificial neural networks
- load balance
- fully distributed
- distributed control
- global knowledge