Checkpoint Merging via Bayesian Optimization in LLM Pretraining.
Deyuan LiuZecheng WangBingning WangWeipeng ChenChunshan LiZhiying TuDianhui ChuBo LiDianbo SuiPublished in: CoRR (2024)
Keyphrases
- optimization algorithm
- efficient optimization
- bayesian networks
- bayesian learning
- optimal design
- optimization method
- optimization problems
- optimization process
- optimization methods
- data sets
- real time
- monte carlo sampling
- optimization model
- constrained optimization
- posterior probability
- load balancing
- model selection
- maximum likelihood
- data driven
- evolutionary algorithm
- information systems
- real world