SpikeBERT: A Language Spikformer Trained with Two-Stage Knowledge Distillation from BERT.
Changze LvTianlong LiJianhan XuChenxi GuZixuan LingCenyuan ZhangXiaoqing ZhengXuanjing HuangPublished in: CoRR (2023)
Keyphrases
- knowledge base
- domain knowledge
- prior knowledge
- knowledge acquisition
- knowledge based systems
- knowledge management
- domain experts
- language learning
- representation language
- conceptual graphs
- natural language
- artificial intelligence
- training set
- knowledge sharing
- linguistic knowledge
- computer programs
- modeling language
- knowledge extraction
- knowledge sources
- conceptual model
- programming language
- knowledge discovery
- knowledge representation
- expert systems