BERT Model Compression With Decoupled Knowledge Distillation And Representation Learning.
Linna ZhangYuehui ChenYi CaoYaou ZhaoPublished in: AISS (2022)
Keyphrases
- prior knowledge
- mathematical model
- learning systems
- formal model
- expert knowledge
- domain knowledge
- learning algorithm
- knowledge acquisition
- conceptual model
- subject matter
- conceptual framework
- neural network
- computational model
- probabilistic model
- structured representations
- learning tasks
- learning process
- learning scheme
- learning models
- long term memory
- additional knowledge
- multiple representations
- inference process
- high level
- learning phase
- qualitative models
- image quality
- concept maps
- objective function
- active learning
- user model
- graphical models
- background knowledge