Heavy-Tail Phenomenon in Decentralized SGD.
Mert GürbüzbalabanYuanhan HuUmut SimsekliKun YuanLingjiong ZhuPublished in: CoRR (2022)
Keyphrases
- cooperative
- multi agent
- peer to peer
- heavy tailed
- distributed systems
- stochastic gradient descent
- extreme value theory
- neural network
- machine learning
- power law
- information flow control
- real time
- security administration
- distributed constraint optimization
- distributed agents
- loss function
- least squares
- dynamic programming
- recommender systems
- decision making
- search engine
- data sets