SAP-SGD: Accelerating Distributed Parallel Training with High Communication Efficiency on Heterogeneous Clusters.
Jing CaoZongwei ZhuXuehai ZhouPublished in: CLUSTER (2021)
Keyphrases
- communication overhead
- spatially distributed
- communication cost
- computer networks
- distributed processing
- stochastic gradient descent
- fully distributed
- distributed environment
- distributed information systems
- parallel execution
- high efficiency
- heterogeneous environments
- distributed control
- clustering algorithm
- loosely coupled
- distributed systems
- heterogeneous systems
- multi party
- master slave
- supervised learning
- load balance
- distributed data
- parallel processing
- feature selection
- single point of failure
- open systems
- heterogeneous networks
- communication networks
- training process
- parallel algorithm
- training speed
- mobile agents
- business intelligence
- online learning