O(1) Communication for Distributed SGD through Two-Level Gradient Averaging.
Subhadeep BhattacharyaWeikuan YuFahim Tahmid ChowdhuryPublished in: CoRR (2020)
Keyphrases
- communication overhead
- communication cost
- distributed control
- distributed systems
- computer networks
- spatially distributed
- fully distributed
- peer to peer
- distributed environment
- distributed computation
- distributed network
- distributed data
- cooperative
- communication networks
- information sharing
- edge detection
- distributed teams
- communication systems
- group communication
- multi agent
- computing environments
- gradient information
- exchange information
- stochastic gradient descent
- fault tolerant
- global knowledge
- single point of failure