Multi-level Distillation of Semantic Knowledge for Pre-training Multilingual Language Model.
Mingqi LiFei DingDan ZhangLong ChengHongxin HuFeng LuoPublished in: CoRR (2022)
Keyphrases
- language model
- semantic knowledge
- language modeling
- n gram
- probabilistic model
- document retrieval
- information retrieval
- cross lingual
- domain knowledge
- knowledge sources
- semantic information
- natural language text
- retrieval model
- query expansion
- training set
- domain ontology
- query terms
- multiword
- smoothing methods
- data mining
- knowledge based systems
- prior knowledge
- search engine