SKDBERT: Compressing BERT via Stochastic Knowledge Distillation.

Zixiang Ding Guoqing Jiang Shuai Zhang Lin Guo Wei Lin

Published in: CoRR (2022)

Keyphrases

domain knowledge
knowledge management
knowledge base
knowledge acquisition
knowledge based systems
learning systems
monte carlo
database systems
knowledge sources
knowledge extraction
information systems
bayesian networks
prior knowledge
domain experts
human experts