Improving Knowledge Distillation for BERT Models: Loss Functions, Mapping Methods, and Weight Tuning.

Apoorv Dankar Adeem Jassani Kartikaeya Kumar

Published in: CoRR (2023)

Keyphrases

loss function
statistical models
machine learning methods
machine learning algorithms
logistic regression
learning models
information retrieval
support vector
pairwise
supervised learning
model selection
cross validation
boosting algorithms