Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time.
Mitchell WortsmanGabriel IlharcoSamir Ya GadreRebecca RoelofsRaphael Gontijo LopesAri S. MorcosHongseok NamkoongAli FarhadiYair CarmonSimon KornblithLudwig SchmidtPublished in: ICML (2022)
Keyphrases
- probabilistic model
- modeling framework
- parameter estimation
- hybrid model
- experimental data
- computational model
- goodness of fit
- models built
- generic model
- learning models
- computational models
- high level
- domain models
- multiple models
- inference process
- neural network model
- feature selection
- prior knowledge
- statistical model
- statistical methods
- mathematical model
- predictive model
- accurate models
- classification accuracy
- maximum likelihood
- prediction accuracy
- process model
- structured prediction
- analytical model
- weighting scheme
- dynamic bayesian networks
- autoregressive
- linear model
- monte carlo simulation
- statistical models
- classification models
- prediction model
- conceptual model