Hanayo: Harnessing Wave-like Pipeline Parallelism for Enhanced Large Model Training Efficiency.

Ziming Liu Shenggan Cheng Haotian Zhou Yang You

Published in: CoRR (2023)

Keyphrases

mathematical model
prior knowledge
experimental data
formal model
probability distribution
database
test set
input data
parallel processing
statistical model
computational model
theoretical analysis
training set
computational complexity
multiscale
high level
feature selection
neural network