2Direction: Theoretically Faster Distributed Training with Bidirectional Communication Compression.
Alexander TyurinPeter RichtárikPublished in: NeurIPS (2023)
Keyphrases
- communication overhead
- communication cost
- distributed systems
- computer networks
- fully distributed
- distributed control
- distributed computation
- spatially distributed
- training set
- open systems
- cooperative
- distributed environment
- training phase
- communication systems
- compression scheme
- data compression
- compression ratio
- peer to peer
- supervised learning
- concurrent processes
- hearing impaired
- multimedia communication
- interprocess communication
- neural network
- exchange information
- multi party
- training algorithm
- computing environments
- lightweight
- image compression
- sensor networks
- multi agent systems
- multi agent
- multiscale