ACCO: Accumulate while you Communicate, Hiding Communications in Distributed LLM Training.
Adel NabliLouis FournierPierre ErbacherLouis SerranoEugene BelilovskyEdouard OyallonPublished in: CoRR (2024)
Keyphrases
- distributed network
- distributed environment
- distributed systems
- group communication
- cooperative
- genetic algorithm
- database
- computer networks
- fault tolerant
- computing environments
- training set
- active learning
- wireless sensor networks
- supervised learning
- information systems
- communication systems
- distributed computing
- distributed processing