Shadowheart SGD: Distributed Asynchronous SGD with Optimal Time Complexity Under Arbitrary Computation and Communication Heterogeneity.
Alexander TyurinMarta PozziIvan IlinPeter RichtárikPublished in: CoRR (2024)
Keyphrases
- worst case
- stochastic gradient descent
- communication overhead
- communication cost
- fully distributed
- distributed systems
- computer networks
- spatially distributed
- distributed control
- cooperative
- asynchronous communication
- optimal solution
- message transmission
- distributed network
- distributed computation
- dynamic programming
- computational complexity
- multi agent
- information dissemination
- communication systems
- peer to peer
- space complexity
- multimedia communication
- least squares
- lower bound
- distributed constraint satisfaction
- single point of failure