O(1) Communication for Distributed SGD through Two-Level Gradient Averaging.
Subhadeep BhattacharyaWeikuan YuFahim Tahmid ChowdhuryKathryn MohrorPublished in: CLUSTER (2021)
Keyphrases
- communication overhead
- communication cost
- spatially distributed
- computer networks
- distributed control
- fully distributed
- distributed systems
- communication networks
- multimedia communication
- single point of failure
- multi agent
- cooperative
- information dissemination
- multi party
- fault tolerant
- stochastic gradient descent
- communication technologies
- communication systems
- distributed environment
- global knowledge
- multi robot coordination
- machine learning
- open systems
- mobile agents
- data sources
- objective function
- multiscale
- decision trees