Lifelong Language Pretraining with Distribution-Specialized Experts.
Wuyang ChenYanqi ZhouNan DuYanping HuangJames LaudonZhifeng ChenClaire CuiPublished in: ICML (2023)
Keyphrases
- natural language
- general purpose
- spatial distribution
- language learning
- programming language
- linguistic knowledge
- domain experts
- english language
- power law
- random variables
- learning activities
- database systems
- knowledge base
- domain specific
- learning scenarios
- data model
- normal distribution
- language processing
- lifelong learning
- high level
- metadata