Sanjiv Kumar

Publication Activity (10 Years)

Years Active: 1999-2024
Publications (10 Years): 173

Top Venues

CoRR

ICLR

NeurIPS

ICML

Publications

Yongchao Zhou, Kaifeng Lyu, Ankit Singh Rawat, Aditya Krishna Menon, Afshin Rostamizadeh, Sanjiv Kumar, Jean-François Kagy, Rishabh Agarwal
DistillSpec: Improving Speculative Decoding via Knowledge Distillation. ICLR (2024)
Sadeep Jayasumana, Srikumar Ramalingam, Andreas Veit, Daniel Glasner, Ayan Chakrabarti, Sanjiv Kumar
Rethinking FID: Towards a Better Evaluation Metric for Image Generation. CoRR (2024)
Abhishek Panigrahi, Nikunj Saunshi, Kaifeng Lyu, Sobhan Miryoosefi, Sashank J. Reddi, Satyen Kale, Sanjiv Kumar
Efficient Stagewise Pretraining via Progressive Subnetworks. CoRR (2024)
Harikrishna Narasimhan, Wittawat Jitkrittum, Ankit Singh Rawat, Seungyeon Kim, Neha Gupta, Aditya Krishna Menon, Sanjiv Kumar
Faster Cascades via Speculative Decoding. CoRR (2024)
Michal Lukasik, Harikrishna Narasimhan, Aditya Krishna Menon, Felix X. Yu, Sanjiv Kumar
Metric-aware LLM inference. CoRR (2024)
Sachin Goyal, Ziwei Ji, Ankit Singh Rawat, Aditya Krishna Menon, Sanjiv Kumar, Vaishnavh Nagarajan
Think before you speak: Training Language Models With Pause Tokens. ICLR (2024)
Neha Gupta, Harikrishna Narasimhan, Wittawat Jitkrittum, Ankit Singh Rawat, Aditya Krishna Menon, Sanjiv Kumar
Language Model Cascades: Token-level uncertainty and beyond. CoRR (2024)
Harikrishna Narasimhan, Aditya Krishna Menon, Wittawat Jitkrittum, Sanjiv Kumar
Plugin estimators for selective classification with out-of-distribution detection. ICLR (2024)
Lin Chen, Michal Lukasik, Wittawat Jitkrittum, Chong You, Sanjiv Kumar
On Bias-Variance Alignment in Deep Models. ICLR (2024)
Shanda Li, Chong You, Guru Guruganesh, Joshua Ainslie, Santiago Ontañón, Manzil Zaheer, Sumit Sanghai, Yiming Yang, Sanjiv Kumar, Srinadh Bhojanapalli
Functional Interpolation for Relative Positions improves Long Context Transformers. ICLR (2024)
Taehyeon Kim, Ananda Theertha Suresh, Kishore Papineni, Michael Riley, Sanjiv Kumar, Adrian Benton
Towards Fast Inference: Exploring and Improving Blockwise Parallel Drafts. CoRR (2024)
Aishwarya P. S, Pranav Ajit Nair, Yashas Samaga, Toby Boyd, Sanjiv Kumar, Prateek Jain, Praneeth Netrapalli
Tandem Transformers for Inference Efficient LLMs. CoRR (2024)
Yuchen Li, Alexandre Kirchmeyer, Aashay Mehta, Yilong Qin, Boris Dadachev, Kishore Papineni, Sanjiv Kumar, Andrej Risteski
Promises and Pitfalls of Generative Masked Language Modeling: Theoretical Framework and Practical Guidelines. CoRR (2024)
Ziwei Ji, Himanshu Jain, Andreas Veit, Sashank J. Reddi, Sadeep Jayasumana, Ankit Singh Rawat, Aditya Krishna Menon, Felix Yu, Sanjiv Kumar
Efficient Document Ranking with Learnable Late Interactions. CoRR (2024)
Ke Ye, Heinrich Jiang, Afshin Rostamizadeh, Ayan Chakrabarti, Giulia DeSalvo, Jean-François Kagy, Lazaros Karydas, Gui Citovsky, Sanjiv Kumar
SpacTor-T5: Pre-training T5 Models with Span Corruption and Replaced Token Detection. CoRR (2024)
Stefani Karp, Nikunj Saunshi, Sobhan Miryoosefi, Sashank J. Reddi, Sanjiv Kumar
Landscape-Aware Growing: The Power of a Little LAG. CoRR (2024)
Yihan Wang, Si Si, Daliang Li, Michal Lukasik, Felix Yu, Cho-Jui Hsieh, Inderjit S. Dhillon, Sanjiv Kumar
Two-stage LLM Fine-tuning with Less Specialization and More Generalization. ICLR (2024)
Yashas Samaga, Varun Yerram, Chong You, Srinadh Bhojanapalli, Sanjiv Kumar, Prateek Jain, Praneeth Netrapalli
HiRE: High Recall Approximate Top-k Estimation for Efficient LLM Inference. CoRR (2024)
Philip Sun, David Simcha, Dave Dopson, Ruiqi Guo, Sanjiv Kumar
SOAR: Improved Indexing for Approximate Nearest Neighbor Search. CoRR (2024)
Neha Gupta, Harikrishna Narasimhan, Wittawat Jitkrittum, Ankit Singh Rawat, Aditya Krishna Menon, Sanjiv Kumar
Language Model Cascades: Token-Level Uncertainty And Beyond. ICLR (2024)
Michal Lukasik, Vaishnavh Nagarajan, Ankit Singh Rawat, Aditya Krishna Menon, Sanjiv Kumar
What do larger image classifiers memorise? Trans. Mach. Learn. Res. 2024 (2024)
Harikrishna Narasimhan, Aditya Krishna Menon, Wittawat Jitkrittum, Neha Gupta, Sanjiv Kumar
Learning to Reject Meets Long-tail Learning. ICLR (2024)
Vaishnavh Nagarajan, Aditya Krishna Menon, Srinadh Bhojanapalli, Hossein Mobahi, Sanjiv Kumar
On student-teacher deviations in distillation: does it pay to disobey? NeurIPS (2023)
Daliang Li, Ankit Singh Rawat, Manzil Zaheer, Xin Wang, Michal Lukasik, Andreas Veit, Felix X. Yu, Sanjiv Kumar
Large Language Models with Controllable Working Memory. ACL (Findings) (2023)
Sadeep Jayasumana, Daniel Glasner, Srikumar Ramalingam, Andreas Veit, Ayan Chakrabarti, Sanjiv Kumar
SPEGTI: Structured Prediction for Efficient Generative Text-to-Image Models. CoRR (2023)
Sashank J. Reddi, Sobhan Miryoosefi, Stefani Karp, Shankar Krishnan, Satyen Kale, Seungyeon Kim, Sanjiv Kumar
Efficient Training of Language Models using Few-Shot Learning. ICML (2023)
Wittawat Jitkrittum, Neha Gupta, Aditya Krishna Menon, Harikrishna Narasimhan, Ankit Singh Rawat, Sanjiv Kumar
When Does Confidence-Based Cascade Deferral Suffice? NeurIPS (2023)
Zonglin Li, Chong You, Srinadh Bhojanapalli, Daliang Li, Ankit Singh Rawat, Sashank J. Reddi, Ke Ye, Felix Chern, Felix X. Yu, Ruiqi Guo, Sanjiv Kumar
The Lazy Neuron Phenomenon: On Emergence of Activation Sparsity in Transformers. ICLR (2023)
Wittawat Jitkrittum, Neha Gupta, Aditya Krishna Menon, Harikrishna Narasimhan, Ankit Singh Rawat, Sanjiv Kumar
When Does Confidence-Based Cascade Deferral Suffice? CoRR (2023)
Philip Sun, Ruiqi Guo, Sanjiv Kumar
Automating Nearest Neighbor Search Configuration with Constrained Optimization. CoRR (2023)
Samy Jelassi, Boris Hanin, Ziwei Ji, Sashank J. Reddi, Srinadh Bhojanapalli, Sanjiv Kumar
Depth Dependence of μP Learning Rates in ReLU MLPs. CoRR (2023)
Yongchao Zhou, Kaifeng Lyu, Ankit Singh Rawat, Aditya Krishna Menon, Afshin Rostamizadeh, Sanjiv Kumar, Jean-François Kagy, Rishabh Agarwal
DistillSpec: Improving Speculative Decoding via Knowledge Distillation. CoRR (2023)
Shanda Li, Chong You, Guru Guruganesh, Joshua Ainslie, Santiago Ontañón, Manzil Zaheer, Sumit Sanghai, Yiming Yang, Sanjiv Kumar, Srinadh Bhojanapalli
Functional Interpolation for Relative Positions Improves Long Context Transformers. CoRR (2023)
Sachin Goyal, Ziwei Ji, Ankit Singh Rawat, Aditya Krishna Menon, Sanjiv Kumar, Vaishnavh Nagarajan
Think before you speak: Training Language Models With Pause Tokens. CoRR (2023)
Gui Citovsky, Giulia DeSalvo, Sanjiv Kumar, Srikumar Ramalingam, Afshin Rostamizadeh, Yunjuan Wang
Leveraging Importance Weights in Subset Selection. ICLR (2023)
Philip Sun, Ruiqi Guo, Sanjiv Kumar
Automating Nearest Neighbor Search Configuration with Constrained Optimization. ICLR (2023)
Seungyeon Kim, Ankit Singh Rawat, Manzil Zaheer, Sadeep Jayasumana, Veeranjaneyulu Sadhanala, Wittawat Jitkrittum, Aditya Krishna Menon, Rob Fergus, Sanjiv Kumar
EmbedDistill: A Geometric Knowledge Distillation for Information Retrieval. CoRR (2023)
Hrayr Harutyunyan, Ankit Singh Rawat, Aditya Krishna Menon, Seungyeon Kim, Sanjiv Kumar
Supervision Complexity and its Role in Knowledge Distillation. CoRR (2023)
Zitong Yang, Michal Lukasik, Vaishnavh Nagarajan, Zonglin Li, Ankit Singh Rawat, Manzil Zaheer, Aditya Krishna Menon, Sanjiv Kumar
ResMem: Learn what you can and memorize the rest. CoRR (2023)
Michal Lukasik, Vaishnavh Nagarajan, Ankit Singh Rawat, Aditya Krishna Menon, Sanjiv Kumar
What do larger image classifiers memorise? CoRR (2023)
Si Si, Felix X. Yu, Ankit Singh Rawat, Cho-Jui Hsieh, Sanjiv Kumar
Serving Graph Compression for Graph Neural Networks. ICLR (2023)
Gui Citovsky, Giulia DeSalvo, Sanjiv Kumar, Srikumar Ramalingam, Afshin Rostamizadeh, Yunjuan Wang
Leveraging Importance Weights in Subset Selection. CoRR (2023)
Guna Shekar M., Wonjun Lee, Sanjiv Kumar, Yanan Duan, Imtiaz Rangwala
A Framework for Developing the Next Generation Interactive Soil Moisture Forecasting System Using the Long-Short Term Memory Model. ICMLA (2023)
Manzil Zaheer, Ankit Singh Rawat, Seungyeon Kim, Chong You, Himanshu Jain, Andreas Veit, Rob Fergus, Sanjiv Kumar
Teacher Guided Training: An Efficient Framework for Knowledge Transfer. ICLR (2023)
Zitong Yang, Michal Lukasik, Vaishnavh Nagarajan, Zonglin Li, Ankit Singh Rawat, Manzil Zaheer, Aditya Krishna Menon, Sanjiv Kumar
ResMem: Learn what you can and memorize the rest. NeurIPS (2023)
Lin Chen, Michal Lukasik, Wittawat Jitkrittum, Chong You, Sanjiv Kumar
It's an Alignment, Not a Trade-off: Revisiting Bias and Variance in Deep Models. CoRR (2023)
Renat Aksitov, Sobhan Miryoosefi, Zonglin Li, Daliang Li, Sheila Babayan, Kavya Kopparapu, Zachary Fisher, Ruiqi Guo, Sushant Prakash, Pranesh Srinivasan, Manzil Zaheer, Felix X. Yu, Sanjiv Kumar
ReST meets ReAct: Self-Improvement for Multi-Step Reasoning LLM Agent. CoRR (2023)
Hrayr Harutyunyan, Ankit Singh Rawat, Aditya Krishna Menon, Seungyeon Kim, Sanjiv Kumar
Supervision Complexity and its Role in Knowledge Distillation. ICLR (2023)
Philip Sun, David Simcha, Dave Dopson, Ruiqi Guo, Sanjiv Kumar
SOAR: Improved Indexing for Approximate Nearest Neighbor Search. NeurIPS (2023)
Harikrishna Narasimhan, Aditya Krishna Menon, Wittawat Jitkrittum, Sanjiv Kumar
Learning to reject meets OOD detection: Are all abstentions created equal? CoRR (2023)
Srikumar Ramalingam, Pranjal Awasthi, Sanjiv Kumar
A Weighted K-Center Algorithm for Data Subset Selection. CoRR (2023)
Vaishnavh Nagarajan, Aditya Krishna Menon, Srinadh Bhojanapalli, Hossein Mobahi, Sanjiv Kumar
On student-teacher deviations in distillation: does it pay to disobey? CoRR (2023)
Harikrishna Narasimhan, Wittawat Jitkrittum, Aditya Krishna Menon, Ankit Singh Rawat, Sanjiv Kumar
Post-hoc estimators for learning to defer to an expert. NeurIPS (2022)
Felix Chern, Blake Hechtman, Andy Davis, Ruiqi Guo, David Majnemer, Sanjiv Kumar
TPU-KNN: K Nearest Neighbor Search at Peak FLOP/s. CoRR (2022)
Zonglin Li, Ruiqi Guo, Sanjiv Kumar
Decoupled Context Processing for Context Augmented Language Modeling. CoRR (2022)
Wittawat Jitkrittum, Aditya Krishna Menon, Ankit Singh Rawat, Sanjiv Kumar
ELM: Embedding and Logit Margins for Long-Tail Learning. CoRR (2022)
Aditya Krishna Menon, Sadeep Jayasumana, Ankit Singh Rawat, Seungyeon Kim, Sashank J. Reddi, Sanjiv Kumar
In defense of dual-encoders for neural ranking. ICML (2022)
Gautam Gautam, Neeraj Sharma, Sanjiv Kumar
Radio over FSO System for 5G Wireless Communication. ICCCNT (2022)
Manzil Zaheer, Ankit Singh Rawat, Seungyeon Kim, Chong You, Himanshu Jain, Andreas Veit, Rob Fergus, Sanjiv Kumar
Teacher Guided Training: An Efficient Framework for Knowledge Transfer. CoRR (2022)
Taman Narayan, Heinrich Jiang, Sen Zhao, Sanjiv Kumar
Predicting on the Edge: Identifying Where a Larger Model Does Better. CoRR (2022)
Arslan Chaudhry, Aditya Krishna Menon, Andreas Veit, Sadeep Jayasumana, Srikumar Ramalingam, Sanjiv Kumar
When does mixup promote local linearity in learned representations? CoRR (2022)
Michal Lukasik, Srinadh Bhojanapalli, Aditya Krishna Menon, Sanjiv Kumar
Teacher's pet: understanding and mitigating biases in distillation. Trans. Mach. Learn. Res. 2022 (2022)
Yihan Wang, Si Si, Daliang Li, Michal Lukasik, Felix X. Yu, Cho-Jui Hsieh, Inderjit S. Dhillon, Sanjiv Kumar
Preserving In-Context Learning ability in Large Language Model Fine-tuning. CoRR (2022)
Zhiyuan Li, Srinadh Bhojanapalli, Manzil Zaheer, Sashank J. Reddi, Sanjiv Kumar
Robust Training of Neural Networks using Scale Invariant Architectures. CoRR (2022)
Zonglin Li, Chong You, Srinadh Bhojanapalli, Daliang Li, Ankit Singh Rawat, Sashank J. Reddi, Ke Ye, Felix X. Chern, Felix X. Yu, Ruiqi Guo, Sanjiv Kumar
Large Models are Parsimonious Learners: Activation Sparsity in Trained Transformers. CoRR (2022)
Zhiyuan Li, Srinadh Bhojanapalli, Manzil Zaheer, Sashank J. Reddi, Sanjiv Kumar
Robust Training of Neural Networks Using Scale Invariant Architectures. ICML (2022)
Daliang Li, Ankit Singh Rawat, Manzil Zaheer, Xin Wang, Michal Lukasik, Andreas Veit, Felix X. Yu, Sanjiv Kumar
Large Language Models with Controllable Working Memory. CoRR (2022)
Felix Chern, Blake Hechtman, Andy Davis, Ruiqi Guo, David Majnemer, Sanjiv Kumar
TPU-KNN: K Nearest Neighbor Search at Peak FLOP/s. NeurIPS (2022)
Zonglin Li, Ruiqi Guo, Sanjiv Kumar
Decoupled Context Processing for Context Augmented Language Modeling. NeurIPS (2022)
Baris Sumengen, Anand Rajagopalan, Gui Citovsky, David Simcha, Olivier Bachem, Pradipta Mitra, Sam Blasiak, Mason Liang, Sanjiv Kumar
Scaling Hierarchical Agglomerative Clustering to Billion-sized Datasets. CoRR (2021)
Gui Citovsky, Giulia DeSalvo, Claudio Gentile, Lazaros Karydas, Anand Rajagopalan, Afshin Rostamizadeh, Sanjiv Kumar
Batch Active Learning at Scale. CoRR (2021)
Seungyeon Kim, Daniel Glasner, Srikumar Ramalingam, Cho-Jui Hsieh, Kishore Papineni, Sanjiv Kumar
Balancing Robustness and Sensitivity using Feature Contrastive Learning. CoRR (2021)
Sashank J. Reddi, Rama Kumar Pasumarthi, Aditya Krishna Menon, Ankit Singh Rawat, Felix X. Yu, Seungyeon Kim, Andreas Veit, Sanjiv Kumar
RankDistil: Knowledge Distillation for Ranking. AISTATS (2021)
Ankit Singh Rawat, Manzil Zaheer, Aditya Krishna Menon, Amr Ahmed, Sanjiv Kumar
When in Doubt, Summon the Titans: Efficient Inference with Large Models. CoRR (2021)
Ankit Singh Rawat, Aditya Krishna Menon, Wittawat Jitkrittum, Sadeep Jayasumana, Felix X. Yu, Sashank J. Reddi, Sanjiv Kumar
Disentangling Sampling and Labeling Bias for Learning in Large-output Spaces. ICML (2021)
Srikumar Ramalingam, Daniel Glasner, Kaushal Patel, Raviteja Vemulapalli, Sadeep Jayasumana, Sanjiv Kumar
Balancing Constraints and Submodularity in Data Subset Selection. CoRR (2021)
Ankit Singh Rawat, Aditya Krishna Menon, Wittawat Jitkrittum, Sadeep Jayasumana, Felix X. Yu, Sashank J. Reddi, Sanjiv Kumar
Disentangling Sampling and Labeling Bias for Learning in Large-Output Spaces. CoRR (2021)
Ankita Verma, Savita, Sanjiv Kumar
Routing Protocols in Delay Tolerant Networks: Comparative and Empirical Analysis. Wirel. Pers. Commun. 118 (1) (2021)
Srinadh Bhojanapalli, Ayan Chakrabarti, Andreas Veit, Michal Lukasik, Himanshu Jain, Frederick Liu, Yin-Wen Chang, Sanjiv Kumar
Leveraging redundancy in attention with Reuse Transformers. CoRR (2021)
Gui Citovsky, Giulia DeSalvo, Claudio Gentile, Lazaros Karydas, Anand Rajagopalan, Afshin Rostamizadeh, Sanjiv Kumar
Batch Active Learning at Scale. NeurIPS (2021)
Michal Lukasik, Srinadh Bhojanapalli, Aditya Krishna Menon, Sanjiv Kumar
Teacher's pet: understanding and mitigating biases in distillation. CoRR (2021)
Cheng-Yu Hsieh, Chih-Kuan Yeh, Xuanqing Liu, Pradeep Kumar Ravikumar, Seungyeon Kim, Sanjiv Kumar, Cho-Jui Hsieh
Evaluations and Methods for Explanation through Robustness Analysis. ICLR (2021)
Srinadh Bhojanapalli, Ayan Chakrabarti, Himanshu Jain, Sanjiv Kumar, Michal Lukasik, Andreas Veit
Eigen Analysis of Self-Attention and its Reconstruction from Partial Computation. CoRR (2021)
Sashank J. Reddi, Zachary Charles, Manzil Zaheer, Zachary Garrett, Keith Rush, Jakub Konecný, Sanjiv Kumar, Hugh Brendan McMahan
Adaptive Federated Optimization. ICLR (2021)
Aditya Krishna Menon, Ankit Singh Rawat, Sanjiv Kumar
Overparameterisation and worst-case generalisation: friend or foe? ICLR (2021)
Jingzhao Zhang, Aditya Krishna Menon, Andreas Veit, Srinadh Bhojanapalli, Sanjiv Kumar, Suvrit Sra
Coping with Label Shift via Distributionally Robust Optimisation. ICLR (2021)
Srinadh Bhojanapalli, Kimberly Wilber, Andreas Veit, Ankit Singh Rawat, Seungyeon Kim, Aditya Krishna Menon, Sanjiv Kumar
On the Reproducibility of Neural Network Predictions. CoRR (2021)
Aditya Krishna Menon, Ankit Singh Rawat, Sashank J. Reddi, Seungyeon Kim, Sanjiv Kumar
A statistical perspective on distillation. ICML (2021)
Aditya Krishna Menon, Sadeep Jayasumana, Ankit Singh Rawat, Himanshu Jain, Andreas Veit, Sanjiv Kumar
Long-tail learning via logit adjustment. ICLR (2021)
Erik Lindgren, Sashank J. Reddi, Ruiqi Guo, Sanjiv Kumar
Efficient Training of Retrieval Models using Negative Cache. NeurIPS (2021)
Chen Zhu, Ankit Singh Rawat, Manzil Zaheer, Srinadh Bhojanapalli, Daliang Li, Felix X. Yu, Sanjiv Kumar
Modifying Memories in Transformer Models. CoRR (2020)
Felix X. Yu, Ankit Singh Rawat, Aditya Krishna Menon, Sanjiv Kumar
Federated Learning with Only Positive Labels. ICML (2020)
Cheng-Yu Hsieh, Chih-Kuan Yeh, Xuanqing Liu, Pradeep Ravikumar, Seungyeon Kim, Sanjiv Kumar, Cho-Jui Hsieh
Evaluations and Methods for Explanation through Robustness Analysis. CoRR (2020)
Jingzhao Zhang, Sai Praneeth Karimireddy, Andreas Veit, Seungyeon Kim, Sashank J. Reddi, Sanjiv Kumar, Suvrit Sra
Why are Adaptive Methods Good for Attention Models? NeurIPS (2020)
Srinadh Bhojanapalli, Chulhee Yun, Ankit Singh Rawat, Sashank J. Reddi, Sanjiv Kumar
Low-Rank Bottleneck in Multi-head Attention Models. CoRR (2020)
Sadeep Jayasumana, Srikumar Ramalingam, Sanjiv Kumar
Kernelized Classification in Deep Networks. CoRR (2020)
Ankit Singh Rawat, Aditya Krishna Menon, Andreas Veit, Felix X. Yu, Sashank J. Reddi, Sanjiv Kumar
Doubly-stochastic mining for heterogeneous retrieval. CoRR (2020)
Michal Lukasik, Srinadh Bhojanapalli, Aditya Krishna Menon, Sanjiv Kumar
Does label smoothing mitigate label noise? CoRR (2020)
Aditya Krishna Menon, Ankit Singh Rawat, Sashank J. Reddi, Seungyeon Kim, Sanjiv Kumar
Why distillation helps: a statistical perspective. CoRR (2020)
Sashank J. Reddi, Zachary Charles, Manzil Zaheer, Zachary Garrett, Keith Rush, Jakub Konecný, Sanjiv Kumar, H. Brendan McMahan
Adaptive Federated Optimization. CoRR (2020)

Sanjiv Kumar.css-eqlu8i{font-size:small;padding-left:4px;}

Publication Activity (10 Years)

Top Topics

Top Venues

Sanjiv Kumar