Fine-tuning Happens in Tiny Subspaces: Exploring Intrinsic Task-specific Subspaces of Pre-trained Language Models.
Zhong ZhangBang LiuJunming ShaoPublished in: CoRR (2023)
Keyphrases
- language model
- fine tuning
- language modeling
- pre trained
- high dimensional data
- n gram
- speech recognition
- test collection
- information retrieval
- language modelling
- probabilistic model
- low dimensional
- document retrieval
- high dimensional
- statistical language models
- query expansion
- retrieval model
- multi modal
- principal component analysis
- data points
- smoothing methods
- document ranking
- language models for information retrieval
- feature space
- training data
- fine tuned
- relevance model
- linear subspace
- training examples