Understanding and Improving Knowledge Distillation for Quantization-Aware Training of Large Transformer Encoders.
Minsoo KimSihwa LeeSukjin HongDu-Seong ChangJungwook ChoiPublished in: CoRR (2022)
Keyphrases
- domain knowledge
- training set
- knowledge based systems
- knowledge representation
- fuzzy logic
- training samples
- acquire knowledge
- human beings
- fault diagnosis
- domain experts
- knowledge acquisition
- online learning
- deeper understanding
- prior knowledge
- knowledge integration
- computational complexity
- genetic algorithm
- instructional technology
- video compression
- data sets
- knowledge sharing
- image compression
- knowledge discovery
- expert systems
- e learning
- data mining