DistilHuBERT: Speech Representation Learning by Layer-wise Distillation of Hidden-unit BERT.

Heng-Jui Chang Shu-Wen Yang Hung-yi Lee

Published in: CoRR (2021)

Keyphrases

learning algorithm
learning process
prior knowledge
learning systems
online learning
machine learning
learning scheme
mobile learning
learning styles
active learning
reinforcement learning
unsupervised learning
pairwise
speech recognition
neural nets
neural network
elementary school