MixLoRA: Enhancing Large Language Models Fine-Tuning with LoRA based Mixture of Experts.
Dengchun LiYingzi MaNaizheng WangZhiyuan ChengLei DuanJie ZuoCal YangMingjie TangPublished in: CoRR (2024)
Keyphrases
- fine tuning
- language model
- mixture model
- agent technology
- language modeling
- probabilistic model
- expert finding
- n gram
- document retrieval
- information retrieval
- retrieval model
- context sensitive
- query expansion
- speech recognition
- language modelling
- fine tuned
- statistical language models
- smoothing methods
- ad hoc information retrieval
- expectation maximization
- test collection
- pseudo relevance feedback
- translation model
- vector space model
- relevance model
- document ranking
- query terms
- document length
- language models for information retrieval
- query specific
- relevant documents
- intelligent agents
- generative model
- multi agent systems