DDK: Dynamic structure pruning based on differentiable search and recursive knowledge distillation for BERT.
Zhou ZhangYang LuTengfei WangXing WeiZhen WeiPublished in: Neural Networks (2024)
Keyphrases
- search space
- search algorithm
- domain knowledge
- effective pruning
- search efficiency
- dynamic environments
- knowledge representation
- learning systems
- search strategy
- search methods
- knowledge base
- additional knowledge
- exhaustive search
- information seeking
- structural information
- knowledge sharing
- background knowledge
- knowledge management
- knowledge discovery
- knowledge sources
- search strategies
- knowledge acquisition
- prior knowledge
- collaborative learning