Sumit Sanghai

Publication Activity (10 Years)

Years Active: 2020-2024
Publications (10 Years): 23

Top Topics

Relative Position

Triangle Inequality

Top Venues

Publications

Shanda Li, Chong You, Guru Guruganesh, Joshua Ainslie, Santiago Ontañón, Manzil Zaheer, Sumit Sanghai, Yiming Yang, Sanjiv Kumar, Srinadh Bhojanapalli
Functional Interpolation for Relative Positions improves Long Context Transformers. ICLR (2024)
Joshua Ainslie, Tao Lei, Michiel de Jong, Santiago Ontañón, Siddhartha Brahma, Yury Zemlyanskiy, David C. Uthus, Mandy Guo, James Lee-Thorp, Yi Tay, Yun-Hsuan Sung, Sumit Sanghai
CoLT5: Faster Long-Range Transformers with Conditional Computation. EMNLP (2023)
Michiel de Jong, Yury Zemlyanskiy, Nicholas FitzGerald, Joshua Ainslie, Sumit Sanghai, Fei Sha, William W. Cohen
Pre-computed memory or on-the-fly encoding? A hybrid approach to retrieval augmentation makes the most of your compute. ICML (2023)
Michiel de Jong, Yury Zemlyanskiy, Joshua Ainslie, Nicholas FitzGerald, Sumit Sanghai, Fei Sha, William W. Cohen
FiDO: Fusion-in-Decoder optimized for stronger performance and faster inference. ACL (Findings) (2023)
Joshua Ainslie, James Lee-Thorp, Michiel de Jong, Yury Zemlyanskiy, Federico Lebrón, Sumit Sanghai
GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints. EMNLP (2023)
Michiel de Jong, Yury Zemlyanskiy, Nicholas FitzGerald, Sumit Sanghai, William W. Cohen, Joshua Ainslie
GLIMMER: generalized late-interaction memory reranker. CoRR (2023)
Shanda Li, Chong You, Guru Guruganesh, Joshua Ainslie, Santiago Ontañón, Manzil Zaheer, Sumit Sanghai, Yiming Yang, Sanjiv Kumar, Srinadh Bhojanapalli
Functional Interpolation for Relative Positions Improves Long Context Transformers. CoRR (2023)
Michiel de Jong, Yury Zemlyanskiy, Nicholas FitzGerald, Joshua Ainslie, Sumit Sanghai, Fei Sha, William Cohen
Pre-computed memory or on-the-fly encoding? A hybrid approach to retrieval augmentation makes the most of your compute. CoRR (2023)
Luke Vilnis, Yury Zemlyanskiy, Patrick Murray, Alexandre Tachard Passos, Sumit Sanghai
Arithmetic Sampling: Parallel Diverse Decoding for Large Language Models. ICML (2023)
Joshua Ainslie, James Lee-Thorp, Michiel de Jong, Yury Zemlyanskiy, Federico Lebrón, Sumit Sanghai
GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints. CoRR (2023)
Joshua Ainslie, Tao Lei, Michiel de Jong, Santiago Ontañón, Siddhartha Brahma, Yury Zemlyanskiy, David C. Uthus, Mandy Guo, James Lee-Thorp, Yi Tay, Yun-Hsuan Sung, Sumit Sanghai
CoLT5: Faster Long-Range Transformers with Conditional Computation. CoRR (2023)
Yury Zemlyanskiy, Michiel de Jong, Luke Vilnis, Santiago Ontañón, William W. Cohen, Sumit Sanghai, Joshua Ainslie
MEMORY-VQ: Compression for Tractable Internet-Scale Memory. CoRR (2023)
Luke Vilnis, Zachary Fisher, Bhargav Kanagal, Patrick Murray, Sumit Sanghai
ImPaKT: A Dataset for Open-Schema Knowledge Base Construction. CoRR (2022)
Li Yang, Qifan Wang, Zac Yu, Anand Kulkarni, Sumit Sanghai, Bin Shu, Jon Elsas, Bhargav Kanagal
MAVE: A Product Dataset for Multi-source Attribute Value Extraction. WSDM (2022)
Yury Zemlyanskiy, Michiel de Jong, Joshua Ainslie, Panupong Pasupat, Peter Shaw, Linlu Qiu, Sumit Sanghai, Fei Sha
Generate-and-Retrieve: Use Your Predictions to Improve Retrieval for Semantic Parsing. COLING (2022)
Michiel de Jong, Yury Zemlyanskiy, Joshua Ainslie, Nicholas FitzGerald, Sumit Sanghai, Fei Sha, William Cohen
FiDO: Fusion-in-Decoder optimized for stronger performance and faster inference. CoRR (2022)
Luke Vilnis, Yury Zemlyanskiy, Patrick Murray, Alexandre Passos, Sumit Sanghai
Arithmetic Sampling: Parallel Diverse Decoding for Large Language Models. CoRR (2022)
Yury Zemlyanskiy, Michiel de Jong, Joshua Ainslie, Panupong Pasupat, Peter Shaw, Linlu Qiu, Sumit Sanghai, Fei Sha
Generate-and-Retrieve: use your predictions to improve retrieval for semantic parsing. CoRR (2022)
Li Yang, Qifan Wang, Zac Yu, Anand Kulkarni, Sumit Sanghai, Bin Shu, Jon Elsas, Bhargav Kanagal
MAVE: A Product Dataset for Multi-source Attribute Value Extraction. CoRR (2021)
Gurmeet Manku, James Lee-Thorp, Bhargav Kanagal, Joshua Ainslie, Jingchen Feng, Zach Pearson, Ebenezer Anjorin, Sudeep Gandhe, Ilya Eckstein, Jim Rosswog, Sumit Sanghai, Michael Pohl, Larry Adams, D. Sivakumar
ShopTalk: A System for Conversational Faceted Search. CoRR (2021)
Joshua Ainslie, Santiago Ontañón, Chris Alberti, Philip Pham, Anirudh Ravula, Sumit Sanghai
ETC: Encoding Long and Structured Data in Transformers. CoRR (2020)
Joshua Ainslie, Santiago Ontañón, Chris Alberti, Vaclav Cvicek, Zachary Fisher, Philip Pham, Anirudh Ravula, Sumit Sanghai, Qifan Wang, Li Yang
ETC: Encoding Long and Structured Inputs in Transformers. EMNLP (1) (2020)
Qifan Wang, Li Yang, Bhargav Kanagal, Sumit Sanghai, D. Sivakumar, Bin Shu, Zac Yu, Jon Elsas
Learning to Extract Attribute Value from Product via Question Answering: A Multi-task Approach. KDD (2020)