Distilhubert: Speech Representation Learning by Layer-Wise Distillation of Hidden-Unit Bert.
Heng-Jui ChangShu-Wen YangHung-yi LeePublished in: ICASSP (2022)
Keyphrases
- learning process
- reinforcement learning
- language acquisition
- learning scheme
- speech recognition
- learning algorithm
- knowledge acquisition
- learning systems
- learning tasks
- pairwise
- prior knowledge
- learning problems
- word processing
- data sets
- learning analytics
- incremental learning
- human computer interaction
- online learning
- pattern recognition