Towards A Unified View of Sparse Feed-Forward Network in Pretraining Large Language Model.
Zeyu LiuTim DettmersXi LinVeselin StoyanovXian LiPublished in: EMNLP (2023)
Keyphrases
- language model
- feed forward
- recurrent networks
- language modeling
- probabilistic model
- n gram
- back propagation
- neural network
- spiking neural networks
- output layer
- neural nets
- speech recognition
- document retrieval
- artificial neural networks
- retrieval model
- query expansion
- language modelling
- hidden layer
- ad hoc information retrieval
- information retrieval
- test collection
- statistical language models
- context sensitive
- feed forward neural networks
- spiking neurons
- mixture model
- activation function
- smoothing methods
- high dimensional
- document ranking
- pseudo relevance feedback
- language model for information retrieval
- query terms
- network structure
- generative model
- language models for information retrieval