Learning Better Masking for Better Language Model Pre-training.

Dongjie Yang Zhuosheng Zhang Hai Zhao

Published in: ACL (1) (2023)

Keyphrases

language model
supervised learning
prior knowledge
learning algorithm
n gram
active learning
speech recognition
information retrieval
probabilistic model
document retrieval
language modeling
language model for information retrieval
hidden markov models
unsupervised learning