James Lee-Thorp

Publication Activity (10 Years)

Years Active: 2021-2024
Publications (10 Years): 14

Top Topics

Top Venues

Publications

Cícero Nogueira dos Santos, James Lee-Thorp, Isaac Noble, Chung-Ching Chang, David C. Uthus
Memory Augmented Language Models through Mixture of Word Experts. NAACL-HLT (2024)
Joshua Ainslie, Tao Lei, Michiel de Jong, Santiago Ontañón, Siddhartha Brahma, Yury Zemlyanskiy, David C. Uthus, Mandy Guo, James Lee-Thorp, Yi Tay, Yun-Hsuan Sung, Sumit Sanghai
CoLT5: Faster Long-Range Transformers with Conditional Computation. EMNLP (2023)
Joshua Ainslie, James Lee-Thorp, Michiel de Jong, Yury Zemlyanskiy, Federico Lebrón, Sumit Sanghai
GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints. EMNLP (2023)
Aran Komatsuzaki, Joan Puigcerver, James Lee-Thorp, Carlos Riquelme Ruiz, Basil Mustafa, Joshua Ainslie, Yi Tay, Mostafa Dehghani, Neil Houlsby
Sparse Upcycling: Training Mixture-of-Experts from Dense Checkpoints. ICLR (2023)
Cícero Nogueira dos Santos, James Lee-Thorp, Isaac Noble, Chung-Ching Chang, David C. Uthus
Memory Augmented Language Models through Mixture of Word Experts. CoRR (2023)
Joshua Ainslie, James Lee-Thorp, Michiel de Jong, Yury Zemlyanskiy, Federico Lebrón, Sumit Sanghai
GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints. CoRR (2023)
Joshua Ainslie, Tao Lei, Michiel de Jong, Santiago Ontañón, Siddhartha Brahma, Yury Zemlyanskiy, David C. Uthus, Mandy Guo, James Lee-Thorp, Yi Tay, Yun-Hsuan Sung, Sumit Sanghai
CoLT5: Faster Long-Range Transformers with Conditional Computation. CoRR (2023)
Aran Komatsuzaki, Joan Puigcerver, James Lee-Thorp, Carlos Riquelme Ruiz, Basil Mustafa, Joshua Ainslie, Yi Tay, Mostafa Dehghani, Neil Houlsby
Sparse Upcycling: Training Mixture-of-Experts from Dense Checkpoints. CoRR (2022)
James Lee-Thorp, Joshua Ainslie, Ilya Eckstein, Santiago Ontañón
FNet: Mixing Tokens with Fourier Transforms. NAACL-HLT (2022)
Adam Roberts, Hyung Won Chung, Anselm Levskaya, Gaurav Mishra, James Bradbury, Daniel Andor, Sharan Narang, Brian Lester, Colin Gaffney, Afroz Mohiuddin, Curtis Hawthorne, Aitor Lewkowycz, Alex Salcianu, Marc van Zee, Jacob Austin, Sebastian Goodman, Livio Baldini Soares, Haitang Hu, Sasha Tsvyashchenko, Aakanksha Chowdhery, Jasmijn Bastings, Jannis Bulian, Xavier Garcia, Jianmo Ni, Andrew Chen, Kathleen Kenealy, Jonathan H. Clark, Stephan Lee, Dan Garrette, James Lee-Thorp, Colin Raffel, Noam Shazeer, Marvin Ritter, Maarten Bosma, Alexandre Passos, Jeremy Maitin-Shepard, Noah Fiedel, Mark Omernick, Brennan Saeta, Ryan Sepassi, Alexander Spiridonov, Joshua Newlan, Andrea Gesmundo
Scaling Up Models and Data with t5x and seqio. CoRR (2022)
James Lee-Thorp, Joshua Ainslie
Sparse Mixers: Combining MoE and Mixing to build a more efficient BERT. CoRR (2022)
James Lee-Thorp, Joshua Ainslie
Sparse Mixers: Combining MoE and Mixing to build a more efficient BERT. EMNLP (Findings) (2022)
James Lee-Thorp, Joshua Ainslie, Ilya Eckstein, Santiago Ontañón
FNet: Mixing Tokens with Fourier Transforms. CoRR (2021)
Gurmeet Manku, James Lee-Thorp, Bhargav Kanagal, Joshua Ainslie, Jingchen Feng, Zach Pearson, Ebenezer Anjorin, Sudeep Gandhe, Ilya Eckstein, Jim Rosswog, Sumit Sanghai, Michael Pohl, Larry Adams, D. Sivakumar
ShopTalk: A System for Conversational Faceted Search. CoRR (2021)