Maximizing Efficiency of Language Model Pre-training for Learning Representation.

Junmo Kang Suwon Shin Jeonghwan Kim Jae-young Jo Sung-Hyon Myaeng

Published in: CoRR (2021)

Keyphrases

language model
language modeling
supervised learning
learning algorithm
document retrieval
probabilistic model
n gram
speech recognition
language modelling
prior knowledge
context sensitive
statistical machine translation