Login / Signup

RegMix: Data Mixture as Regression for Language Model Pre-training.

Qian LiuXiaosen ZhengNiklas MuennighoffGuangtao ZengLongxu DouTianyu PangJing JiangMin Lin
Published in: CoRR (2024)
Keyphrases
  • language model
  • mixture model
  • n gram
  • training data
  • query processing
  • probabilistic model
  • speech recognition
  • retrieval model
  • context sensitive
  • statistical language models