Fine-tuning Happens in Tiny Subspaces: Exploring Intrinsic Task-specific Subspaces of Pre-trained Language Models.
Zhong ZhangBang LiuJunming ShaoPublished in: ACL (1) (2023)
Keyphrases
- language model
- fine tuning
- language modeling
- pre trained
- n gram
- speech recognition
- probabilistic model
- high dimensional data
- language modelling
- low dimensional
- document retrieval
- principal component analysis
- test collection
- statistical language models
- retrieval model
- smoothing methods
- information retrieval
- high dimensional
- query expansion
- prior knowledge
- feature space
- training data
- data points
- linear subspace
- language models for information retrieval
- information retrieval systems
- document ranking
- control signals
- machine learning