PipeTransformer: Automated Elastic Pipelining for Distributed Training of Transformers.
Chaoyang HeShen LiMahdi SoltanolkotabiSalman AvestimehrPublished in: CoRR (2021)
Keyphrases
- distributed environment
- distributed systems
- fully automated
- multi agent
- semi automated
- training samples
- decision trees
- communication overhead
- automated analysis
- agent technology
- communication cost
- distributed learning
- neural network
- training phase
- distributed computing
- training process
- fault tolerant
- computer networks
- semi automatic
- test set
- training examples
- lightweight
- semi supervised
- search algorithm
- knowledge base