Dependency-Aware Semi-Structured Sparsity of GLU Variants in Large Language Models.
Zhiyu GuoHidetaka KamigaitoTaro WanatnabePublished in: CoRR (2024)
Keyphrases
- language model
- structured sparsity
- language modeling
- probabilistic model
- statistical learning
- n gram
- learning problems
- compressive sensing
- group lasso
- query expansion
- information retrieval
- relevance model
- smoothing methods
- variable selection
- language models for information retrieval
- regression model
- regularization methods
- convex optimization problems
- pattern recognition
- translation model
- sparse representation
- decision trees