Domain Knowledge Transferring for Pre-trained Language Model via Calibrated Activation Boundary Distillation.
Dongha ChoiHongseok ChoiHyunju LeePublished in: ACL (1) (2022)
Keyphrases
- language model
- pre trained
- domain knowledge
- training data
- language modeling
- n gram
- document retrieval
- information retrieval
- probabilistic model
- speech recognition
- test collection
- training examples
- query expansion
- retrieval model
- smoothing methods
- control signals
- prior knowledge
- relevance model
- ad hoc information retrieval
- mixture model
- transfer learning
- translation model
- context sensitive
- small number