Efficient Online Data Mixing For Language Model Pre-Training.

Alon Albalak Liangming Pan Colin Raffel William Yang Wang

Published in: CoRR (2023)

Keyphrases

language model
language modeling
probabilistic model
n gram
retrieval model
information retrieval
information extraction
supervised learning
co occurrence
speech recognition
uncertain data
discrete data