PipePar: Enabling fast DNN pipeline parallel training in heterogeneous GPU clusters.
Jinghui ZhangGeng NiuQiangsheng DaiHaorui LiZhihua WuFang DongZhiang WuPublished in: Neurocomputing (2023)
Keyphrases
- training process
- parallel implementation
- parallel processing
- parallel programming
- parallel computation
- graphics processing units
- parallel computing
- parallel architecture
- cluster of workstations
- clustering algorithm
- parallel hardware
- data clustering
- training set
- training phase
- supervised learning
- real time
- graphics processors
- parallel architectures
- parallel algorithm
- training algorithm
- cluster analysis
- input data
- online learning
- data points
- distributed memory
- test set
- data distribution
- heterogeneous systems
- gpu implementation
- hierarchical clustering
- document clustering