Hanayo: Harnessing Wave-like Pipeline Parallelism for Enhanced Large Model Training Efficiency.
Ziming LiuShenggan ChengHaotian ZhouYang YouPublished in: SC (2023)
Keyphrases
- probabilistic model
- computational complexity
- statistical model
- computational model
- experimental data
- em algorithm
- management system
- structured prediction
- training algorithm
- formal model
- mathematical model
- parameter estimation
- information retrieval
- probability distribution
- training set
- objective function
- multiscale
- similarity measure
- high level