Emergence of a High-Dimensional Abstraction Phase in Language Transformers.
Emily ChengDiego DoimoCorentin KervadecIuri MacoccoJade YuAlessandro LaioMarco BaroniPublished in: CoRR (2024)
Keyphrases
- high dimensional
- partial discharge
- language learning
- programming language
- data abstraction
- similarity search
- high level
- language processing
- natural language
- low dimensional
- parameter space
- data points
- sparse data
- nearest neighbor
- metric space
- input space
- target language
- information retrieval
- microarray data
- noisy data
- variable selection
- collective intelligence
- specification language
- operational semantics