Sign in

METRO: Efficient Denoising Pretraining of Large Scale Autoencoding Language Models with Model Generated Signals.

Payal BajajChenyan XiongGuolin KeXiaodong LiuDi HeSaurabh TiwaryTie-Yan LiuPaul BennettXia SongJianfeng Gao
Published in: CoRR (2022)
Keyphrases
  • language model
  • probabilistic model
  • denoising
  • language modeling
  • bayesian networks
  • text classification
  • generative model
  • error rate
  • n gram
  • statistical model
  • document retrieval