Towards A Unified View of Sparse Feed-Forward Network in Pretraining Large Language Model.
Leo Z. LiuTim DettmersXi Victoria LinVeselin StoyanovXian LiPublished in: CoRR (2023)
Keyphrases
- language model
- feed forward
- language modeling
- recurrent networks
- back propagation
- n gram
- spiking neural networks
- artificial neural networks
- speech recognition
- neural nets
- document retrieval
- spiking neurons
- neural network
- probabilistic model
- output layer
- statistical language models
- language modelling
- hidden layer
- mixture model
- context sensitive
- information retrieval
- query expansion
- feed forward neural networks
- retrieval model
- ad hoc information retrieval
- activation function
- relevance model
- document ranking
- machine learning
- query terms
- high dimensional
- translation model
- network structure
- smoothing methods
- test collection
- search engine
- artificial intelligence
- language models for information retrieval
- language model for information retrieval