oBERTa: Improving Sparse Transfer Learning via improved initialization, distillation, and pruning regimes.
Daniel CamposAlexandre MarquesMark KurtzChengXiang ZhaiPublished in: CoRR (2023)
Keyphrases
- transfer learning
- knowledge transfer
- learning tasks
- labeled data
- reinforcement learning
- active learning
- transfer knowledge
- cross domain
- multi task
- semi supervised learning
- structure learning
- machine learning algorithms
- domain adaptation
- machine learning
- text classification
- collaborative filtering
- text categorization
- high dimensional
- manifold alignment
- cross domain learning
- learning algorithm
- text mining
- neural network
- k means
- bayesian networks
- data mining
- transferring knowledge