Insights into the mechanism behind reusing Teacher's classifier in Knowledge Distillation.

Published in: Tiny Papers @ ICLR (2023)

Keyphrases

training data
domain knowledge
knowledge acquisition
subject matter
feature space
background knowledge
training samples
support vector machine
classification algorithm
domain experts
class labels
learning systems
neural network
prior knowledge
expert systems
decision trees
e learning
learning algorithm