Merak: An Efficient Distributed DNN Training Framework With Automated 3D Parallelism for Giant Foundation Models.
Zhiquan LaiShengwei LiXudong TangKeshi GeWeijie LiuYabo DuanLinbo QiaoDongsheng LiPublished in: IEEE Trans. Parallel Distributed Syst. (2023)
Keyphrases
- probabilistic model
- modeling framework
- lightweight
- distributed environment
- mathematical framework
- statistical model
- distributed learning
- structured prediction
- main contribution
- training process
- bayesian framework
- statistical models
- conditional random fields
- massively parallel
- experimental data
- data sets
- complex systems
- semi supervised
- active learning
- training set
- bayesian networks
- learning algorithm