SlowMo: Improving Communication-Efficient Distributed SGD with Slow Momentum.

Jianyu Wang Vinayak Tantia Nicolas Ballas Michael G. Rabbat

Published in: ICLR (2020)

Keyphrases

communication overhead
communication cost
cooperative
distributed systems
multi agent
lightweight
distributed environment
computer networks
distributed network
spatially distributed
neural network
peer to peer
communication networks
parallel execution
distributed control
global knowledge