PipeTransformer: Automated Elastic Pipelining for Distributed Training of Large-scale Models.
Chaoyang HeShen LiMahdi SoltanolkotabiSalman AvestimehrPublished in: ICML (2021)
Keyphrases
- complex systems
- real world
- statistical models
- probabilistic model
- accurate models
- digital libraries
- data intensive
- neural network
- fine grain
- structured prediction
- mathematical models
- small scale
- fully automated
- computer networks
- distributed environment
- learning algorithm
- experimental data
- data mining
- test set
- model selection
- genetic algorithm
- multi agent
- case study