DoReMi: Optimizing Data Mixtures Speeds Up Language Model Pretraining.

Sang Michael Xie Hieu Pham Xuanyi Dong Nan Du Hanxiao Liu Yifeng Lu Percy S. Liang Quoc V. Le Tengyu Ma Adams Wei Yu

Published in: NeurIPS (2023)

Keyphrases