On Distributed Training of Foundation Models: Challenges and Observations.

Supriyo Chakraborty

Published in: IPDPS Workshops (2023)

Keyphrases

structured prediction
statistical models
cooperative
probabilistic model
mathematical models
fault tolerant
test set
neural network model
complex systems
training process
computing environments
lessons learned
neural network
lightweight
distributed systems
supervised learning
prior knowledge
information systems