Zero Bubble (Almost) Pipeline Parallelism.
Penghui QiXinyi WanGuangxing HuangMin LinPublished in: ICLR (2024)
Keyphrases
- parallel processing
- parallel execution
- massively parallel
- parallel computing
- computational power
- shared memory
- pipeline architecture
- artificial neural networks
- finite element
- parallel computation
- coarse grain
- computer vision
- video sequences
- machine learning
- data structure
- multi agent
- clustering algorithm
- parallel computers
- e learning
- fine grain
- data parallelism