Talking Heads: Understanding Inter-layer Communication in Transformer Language Models.
Jack MerulloCarsten EickhoffEllie PavlickPublished in: CoRR (2024)
Keyphrases
- language model
- inter layer
- language modeling
- n gram
- speech recognition
- probabilistic model
- document retrieval
- retrieval model
- statistical language models
- language modelling
- smoothing methods
- document ranking
- query expansion
- information retrieval
- test collection
- single layer
- language models for information retrieval
- base layer
- scalable video coding
- fuzzy logic
- multiresolution
- feature selection