Understanding and Improving Knowledge Distillation for Quantization Aware Training of Large Transformer Encoders.
Minsoo KimSihwa LeeSukjin HongDu-Seong ChangJungwook ChoiPublished in: EMNLP (2022)
Keyphrases
- domain knowledge
- knowledge base
- knowledge acquisition
- knowledge discovery
- knowledge management
- knowledge based systems
- deeper understanding
- learning systems
- acquired knowledge
- data mining techniques
- knowledge integration
- background knowledge
- knowledge structures
- test set
- solving problems
- training process
- knowledge sharing
- data sets
- training samples
- neural network