Learning Better Masking for Better Language Model Pre-training.

Dongjie Yang Zhuosheng Zhang Hai Zhao

Published in: CoRR (2022)

Keyphrases

language model
n gram
learning algorithm
supervised learning
language modeling
active learning
speech recognition
information retrieval
document retrieval
test collection
statistical language models
language modelling
query terms
document collections
mixture model
text mining
probabilistic model
prior knowledge