AccTFM: An Effective Intra-Layer Model Parallelization Strategy for Training Large-Scale Transformer-Based Models.
Zihao ZengChubo LiuZhuo TangKenli LiKeqin LiPublished in: IEEE Trans. Parallel Distributed Syst. (2022)
Keyphrases
- probabilistic model
- accurate models
- statistical models
- statistical model
- hybrid model
- classification models
- parameter estimation
- computational model
- autoregressive
- experimental data
- computational models
- modelling language
- structured prediction
- linear models
- parametric models
- model construction
- fuzzy logic
- linear model
- generic model
- data sets
- neural network model
- conceptual model
- em algorithm
- mathematical model
- real world
- hierarchical model
- analytical model
- objective function
- predictive model
- model fitting
- prior knowledge
- training algorithm
- mathematical models
- learning models
- supervised learning
- expectation maximization
- complex systems