Multi-Level Local SGD: Distributed SGD for Heterogeneous Hierarchical Networks.
Timothy CastigliaAnirban DasStacy PattersonPublished in: ICLR (2021)
Keyphrases
- stochastic gradient descent
- heterogeneous networks
- distributed systems
- computer networks
- loosely coupled
- hierarchical architecture
- peer to peer networks
- structured peer to peer
- distributed information systems
- heterogeneous environments
- distributed data
- loss function
- multi agent
- hierarchical structure
- network structure
- query decomposition
- network nodes
- wide area network
- transparent access
- distributed heterogeneous
- heterogeneous data
- computing environments
- distributed environment
- linear combination
- heterogeneous databases
- heterogeneous data sources
- random forests
- multi layer
- step size
- distributed search
- distributed network
- fault tolerant
- peer to peer
- least squares
- sensor networks
- cooperative