Cupcake: A Compression Scheduler for Scalable Communication-Efficient Distributed Training.
Zhuang WangXinyu Crystal WuZhaozhuo XuT. S. Eugene NgPublished in: MLSys (2023)
Keyphrases
- lightweight
- fully distributed
- communication overhead
- communication cost
- distributed control
- cooperative
- supervised learning
- distributed systems
- highly scalable
- neural network
- spatially distributed
- distributed network
- highly efficient
- training samples
- information sharing
- training examples
- parallel execution
- distributed computation
- group communication
- peer to peer
- multi agent
- data sets
- scalable distributed