Lightweight Model Pre-training via Language Guided Knowledge Distillation.
Mingsheng LiLin ZhangMingzhen ZhuZilong HuangGang YuJiayuan FanTao ChenPublished in: CoRR (2024)
Keyphrases
- lightweight
- conceptual model
- prior knowledge
- formal model
- high level
- probabilistic model
- bayesian networks
- training algorithm
- domain knowledge
- metamodel
- expert knowledge
- computational model
- mathematical model
- semantic models
- neural network
- formal representation
- language learning
- knowledge acquisition
- knowledge management
- knowledge representation
- expert systems
- objective function
- training data