Multi-level Distillation of Semantic Knowledge for Pre-training Multilingual Language Model.
Mingqi LiFei DingDan ZhangLong ChengHongxin HuFeng LuoPublished in: EMNLP (2022)
Keyphrases
- language model
- semantic knowledge
- language modeling
- n gram
- document retrieval
- probabilistic model
- semantic information
- domain knowledge
- cross lingual
- information retrieval
- retrieval model
- knowledge sources
- training set
- natural language text
- query terms
- smoothing methods
- multiword
- domain ontology
- query expansion
- cross language information retrieval
- translation model
- high level
- visual features
- knowledge based systems
- supervised learning
- knowledge discovery
- bayesian networks