Nathan Godey

Publication Activity (10 Years)

Years Active: 2022-2024
Publications (10 Years): 10

Top Topics

Query Expansion

Text Localization And Recognition

Retrieval Model

Top Venues

Publications

Nathan Godey, Éric de la Clergerie, Benoît Sagot
Anisotropy Is Inherent to Self-Attention in Transformers. CoRR (2024)
Nathan Godey, Éric Villemonte de la Clergerie, Benoît Sagot
Anisotropy Is Inherent to Self-Attention in Transformers. EACL (1) (2024)
Nathan Godey, Éric de la Clergerie, Benoît Sagot
Why do small language models underperform? Studying Language Model Saturation via the Softmax Bottleneck. CoRR (2024)
Nathan Godey, Éric Villemonte de la Clergerie, Benoît Sagot
Headless Language Models: Learning without Predicting with Contrastive Weight Tying. ICLR (2024)
Nathan Godey, Éric de la Clergerie, Benoît Sagot
On the Scaling Laws of Geographical Representation in Language Models. CoRR (2024)
Nathan Godey, Éric de la Clergerie, Benoît Sagot
On the Scaling Laws of Geographical Representation in Language Models. LREC/COLING (2024)
Nathan Godey, Éric de la Clergerie, Benoît Sagot
Is Anisotropy Inherent to Transformers? CoRR (2023)
Nathan Godey, Éric de la Clergerie, Benoît Sagot
Headless Language Models: Learning without Predicting with Contrastive Weight Tying. CoRR (2023)
Nathan Godey, Roman Castagné, Éric de la Clergerie, Benoît Sagot
MANTa: Efficient Gradient-Based Tokenization for Robust End-to-End Language Modeling. CoRR (2022)
Nathan Godey, Roman Castagné, Éric de la Clergerie, Benoît Sagot
MANTa: Efficient Gradient-Based Tokenization for End-to-End Robust Language Modeling. EMNLP (Findings) (2022)