Beidi Chen

Publication Activity (10 Years)

Years Active: 2016-2024
Publications (10 Years): 73

Top Topics

Nearest Neighbor

Gradient Estimation

Top Venues

Publications

Mostafa Elhoushi, Akshat Shrivastava, Diana Liskovich, Basil Hosmer, Bram Wasti, Liangzhen Lai, Anas Mahmoud, Bilge Acun, Saurabh Agarwal, Ahmed Roman, Ahmed A Aly, Beidi Chen, Carole-Jean Wu
LayerSkip: Enabling Early Exit Inference and Self-Speculative Decoding. CoRR (2024)
Zhihang Yuan, Yuzhang Shang, Yang Zhou, Zhen Dong, Zhe Zhou, Chenhao Xue, Bingzhe Wu, Zhikai Li, Qingyi Gu, Yong Jae Lee, Yan Yan, Beidi Chen, Guangyu Sun, Kurt Keutzer
LLM Inference Unveiled: Survey and Roofline Model Insights. CoRR (2024)
Harry Dong, Xinyu Yang, Zhenyu Zhang, Zhangyang Wang, Yuejie Chi, Beidi Chen
Get More with LESS: Synthesizing Recurrence with KV Cache Compression for Efficient LLM Inference. CoRR (2024)
Cheng Luo, Jiawei Zhao, Zhuoming Chen, Beidi Chen, Anima Anandkumar
MINI-SEQUENCE TRANSFORMER: Optimizing Intermediate Memory for Long Sequences Training. CoRR (2024)
Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, Mike Lewis
Efficient Streaming Language Models with Attention Sinks. ICLR (2024)
Wentao Guo, Jikai Long, Yimeng Zeng, Zirui Liu, Xinyu Yang, Yide Ran, Jacob R. Gardner, Osbert Bastani, Christopher De Sa, Xiaodong Yu, Beidi Chen, Zhaozhuo Xu
Zeroth-Order Fine-Tuning of LLMs with Extreme Sparsity. CoRR (2024)
Ruisi Cai, Yuandong Tian, Zhangyang Wang, Beidi Chen
LoCoCo: Dropping In Convolutions for Long Context Compression. CoRR (2024)
Jiawei Zhao, Zhenyu Zhang, Beidi Chen, Zhangyang Wang, Anima Anandkumar, Yuandong Tian
GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection. CoRR (2024)
Taiming Lu, Lingfeng Shen, Xinyu Yang, Weiting Tan, Beidi Chen, Huaxiu Yao
It Takes Two: On the Seamlessness between Reward and Policy Model in RLHF. CoRR (2024)
Haizhong Zheng, Xiaoyan Bai, Beidi Chen, Fan Lai, Atul Prakash
Learn To be Efficient: Build Structured Sparsity in Large Language Models. CoRR (2024)
Zhuoming Chen, Avner May, Ruslan Svirschevski, Yuhsun Huang, Max Ryabinin, Zhihao Jia, Beidi Chen
Sequoia: Scalable, Robust, and Hardware-aware Speculative Decoding. CoRR (2024)
Mostafa Elhoushi, Akshat Shrivastava, Diana Liskovich, Basil Hosmer, Bram Wasti, Liangzhen Lai, Anas Mahmoud, Bilge Acun, Saurabh Agarwal, Ahmed Roman, Ahmed A Aly, Beidi Chen, Carole-Jean Wu
LayerSkip: Enabling Early Exit Inference and Self-Speculative Decoding. ACL (1) (2024)
Zirui Liu, Jiayi Yuan, Hongye Jin, Shaochen Zhong, Zhaozhuo Xu, Vladimir Braverman, Beidi Chen, Xia Hu
KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache. CoRR (2024)
Xuezhe Ma, Xiaomeng Yang, Wenhan Xiong, Beidi Chen, Lili Yu, Hao Zhang, Jonathan May, Luke Zettlemoyer, Omer Levy, Chunting Zhou
Megalodon: Efficient LLM Pretraining and Inference with Unlimited Context Length. CoRR (2024)
Harry Dong, Beidi Chen, Yuejie Chi
Prompt-prompted Mixture of Experts for Efficient LLM Generation. CoRR (2024)
Zhenyu Zhang, Runjin Chen, Shiwei Liu, Zhewei Yao, Olatunji Ruwase, Beidi Chen, Xiaoxia Wu, Zhangyang Wang
Found in the Middle: How Language Models Use Long Contexts Better via Plug-and-Play Positional Encoding. CoRR (2024)
Zhenyu Zhang, Shiwei Liu, Runjin Chen, Bhavya Kailkhura, Beidi Chen, Atlas Wang
Q-Hitter: A Better Token Oracle for Efficient LLM Inference via Sparse-Quantized KV Cache. MLSys (2024)
Jianyu Zhang, Niklas Nolte, Ranajoy Sadhukhan, Beidi Chen, Léon Bottou
Memory Mosaics. CoRR (2024)
Ruslan Svirschevski, Avner May, Zhuoming Chen, Beidi Chen, Zhihao Jia, Max Ryabinin
SpecExec: Massively Parallel Speculative Decoding for Interactive LLM Inference on Consumer Devices. CoRR (2024)
Hanshi Sun, Zhuoming Chen, Xinyu Yang, Yuandong Tian, Beidi Chen
TriForce: Lossless Acceleration of Long Sequence Generation with Hierarchical Speculative Decoding. CoRR (2024)
Ceyu Xu, Yongji Wu, Xinyu Yang, Beidi Chen, Matthew Lentz, Danyang Zhuo, Lisa Wu Wills
VcLLM: Video Codecs are Secretly Tensor Codecs. CoRR (2024)
Yuandong Tian, Yiping Wang, Zhenyu Zhang, Beidi Chen, Simon Shaolei Du
JoMA: Demystifying Multilayer Transformers via Joint Dynamics of MLP and Attention. ICLR (2024)
Minghan Li, Xilun Chen, Ari Holtzman, Beidi Chen, Jimmy Lin, Wen-tau Yih, Xi Victoria Lin
Nearest Neighbor Speculative Decoding for LLM Generation and Attribution. CoRR (2024)
Zichang Liu, Jue Wang, Tri Dao, Tianyi Zhou, Binhang Yuan, Zhao Song, Anshumali Shrivastava, Ce Zhang, Yuandong Tian, Christopher Ré, Beidi Chen
Deja Vu: Contextual Sparsity for Efficient LLMs at Inference Time. ICML (2023)
Zhenyu Zhang, Ying Sheng, Tianyi Zhou, Tianlong Chen, Lianmin Zheng, Ruisi Cai, Zhao Song, Yuandong Tian, Christopher Ré, Clark W. Barrett, Zhangyang Wang, Beidi Chen
O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models. CoRR (2023)
Ying Sheng, Lianmin Zheng, Binhang Yuan, Zhuohan Li, Max Ryabinin, Daniel Y. Fu, Zhiqiang Xie, Beidi Chen, Clark W. Barrett, Joseph E. Gonzalez, Percy Liang, Christopher Ré, Ion Stoica, Ce Zhang
High-throughput Generative Inference of Large Language Models with a Single GPU. CoRR (2023)
Jue Wang, Yucheng Lu, Binhang Yuan, Beidi Chen, Percy Liang, Christopher De Sa, Christopher Ré, Ce Zhang
CocktailSGD: Fine-tuning Foundation Models over 500Mbps Networks. ICML (2023)
Jiawei Zhao, Yifei Zhang, Beidi Chen, Florian Schäfer, Anima Anandkumar
InRank: Incremental Low-Rank Learning. CoRR (2023)
Stefano Massaroli, Michael Poli, Daniel Y. Fu, Hermann Kumbong, Rom N. Parnichkun, Aman Timalsina, David W. Romero, Quinn McIntyre, Beidi Chen, Atri Rudra, Ce Zhang, Christopher Ré, Stefano Ermon, Yoshua Bengio
Laughing Hyena Distillery: Extracting Compact Recurrences From Convolutions. CoRR (2023)
Yuandong Tian, Yiping Wang, Zhenyu Zhang, Beidi Chen, Simon S. Du
JoMA: Demystifying Multilayer Transformers via JOint Dynamics of MLP and Attention. CoRR (2023)
Moses Charikar, Beidi Chen, Christopher Ré, Erik Waingarten
Fast Algorithms for a New Relaxation of Optimal Transport. CoRR (2023)
Yuandong Tian, Yiping Wang, Beidi Chen, Simon S. Du
Scan and Snap: Understanding Training Dynamics and Token Composition in 1-layer Transformer. CoRR (2023)
Ying Sheng, Lianmin Zheng, Binhang Yuan, Zhuohan Li, Max Ryabinin, Beidi Chen, Percy Liang, Christopher Ré, Ion Stoica, Ce Zhang
FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPU. ICML (2023)
Youhe Jiang, Ran Yan, Xiaozhe Yao, Beidi Chen, Binhang Yuan
HexGen: Generative Inference of Foundation Model over Heterogeneous Decentralized Environment. CoRR (2023)
Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, Mike Lewis
Efficient Streaming Language Models with Attention Sinks. CoRR (2023)
Moses Charikar, Beidi Chen, Christopher Ré, Erik Waingarten
Fast Algorithms for a New Relaxation of Optimal Transport. COLT (2023)
Zhaozhuo Xu, Zirui Liu, Beidi Chen, Yuxin Tang, Jue Wang, Kaixiong Zhou, Xia Hu, Anshumali Shrivastava
Compress, Then Prompt: Improving Accuracy-Efficiency Trade-off of LLM Inference with Transferable Prompt. CoRR (2023)
Andrew Cohen, Weiping Dou, Jiang Zhu, Slawomir Koziel, Peter Renner, Jan-Ove Mattsson, Xiaomeng Yang, Beidi Chen, Kevin Stone, Yuandong Tian
Modeling Scattering Coefficients using Self-Attentive Complex Polynomials with Image-based Representation. CoRR (2023)
Zichang Liu, Jue Wang, Tri Dao, Tianyi Zhou, Binhang Yuan, Zhao Song, Anshumali Shrivastava, Ce Zhang, Yuandong Tian, Christopher Ré, Beidi Chen
Deja Vu: Contextual Sparsity for Efficient LLMs at Inference Time. CoRR (2023)
Zhenyu Zhang, Ying Sheng, Tianyi Zhou, Tianlong Chen, Lianmin Zheng, Ruisi Cai, Zhao Song, Yuandong Tian, Christopher Ré, Clark W. Barrett, Zhangyang Wang, Beidi Chen
H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models. NeurIPS (2023)
Yuandong Tian, Yiping Wang, Beidi Chen, Simon S. Du
Scan and Snap: Understanding Training Dynamics and Token Composition in 1-layer Transformer. NeurIPS (2023)
Stefano Massaroli, Michael Poli, Daniel Y. Fu, Hermann Kumbong, Rom N. Parnichkun, David W. Romero, Aman Timalsina, Quinn McIntyre, Beidi Chen, Atri Rudra, Ce Zhang, Christopher Ré, Stefano Ermon, Yoshua Bengio
Laughing Hyena Distillery: Extracting Compact Recurrences From Convolutions. NeurIPS (2023)
Zichang Liu, Zhaozhuo Xu, Alan Baonan Ji, Junyan Zhang, Jonathan Li, Beidi Chen, Anshumali Shrivastava
HALOS: Hashing Large Output Space for Cheap Inference. MLSys (2022)
Binhang Yuan, Yongjun He, Jared Davis, Tianyi Zhang, Tri Dao, Beidi Chen, Percy Liang, Christopher Ré, Ce Zhang
Decentralized Training of Foundation Models in Heterogeneous Environments. NeurIPS (2022)
Beidi Chen, Tri Dao, Kaizhao Liang, Jiaming Yang, Zhao Song, Atri Rudra, Christopher Ré
Pixelated Butterfly: Simple and Efficient Sparse training for Neural Network Models. ICLR (2022)
Binhang Yuan, Yongjun He, Jared Quincy Davis, Tianyi Zhang, Tri Dao, Beidi Chen, Percy Liang, Christopher Ré, Ce Zhang
Decentralized Training of Foundation Models in Heterogeneous Environments. CoRR (2022)
Tri Dao, Beidi Chen, Nimit Sharad Sohoni, Arjun D. Desai, Michael Poli, Jessica Grogan, Alexander Liu, Aniruddh Rao, Atri Rudra, Christopher Ré
Monarch: Expressive Structured Matrices for Efficient and Accurate Training. ICML (2022)
Jue Wang, Binhang Yuan, Luka Rimanic, Yongjun He, Tri Dao, Beidi Chen, Christopher Ré, Ce Zhang
Fine-tuning Language Models over Slow Networks using Activation Quantization with Guarantees. NeurIPS (2022)
Jue Wang, Binhang Yuan, Luka Rimanic, Yongjun He, Tri Dao, Beidi Chen, Christopher Ré, Ce Zhang
Fine-tuning Language Models over Slow Networks using Activation Compression with Guarantees. CoRR (2022)
Tri Dao, Beidi Chen, Nimit Sharad Sohoni, Arjun D. Desai, Michael Poli, Jessica Grogan, Alexander Liu, Aniruddh Rao, Atri Rudra, Christopher Ré
Monarch: Expressive Structured Matrices for Efficient and Accurate Training. CoRR (2022)
Zhaozhuo Xu, Alan Baonan Ji, Andrew Woods, Beidi Chen, Anshumali Shrivastava
Satellite Images and Deep Learning to Identify Discrepancy in Mailing Addresses with Applications to Census 2020 in Houston. CoRR (2021)
Zhaozhuo Xu, Beidi Chen, Chaojian Li, Weiyang Liu, Le Song, Yingyan Lin, Anshumali Shrivastava
Locality Sensitive Teaching. NeurIPS (2021)
Beidi Chen, Tri Dao, Eric Winsor, Zhao Song, Atri Rudra, Christopher Ré
Scatterbrain: Unifying Sparse and Low-rank Attention. NeurIPS (2021)
Tharun Medini, Beidi Chen, Anshumali Shrivastava
SOLAR: Sparse Orthogonal Learned and Random Embeddings. ICLR (2021)
Beidi Chen, Tri Dao, Eric Winsor, Zhao Song, Atri Rudra, Christopher Ré
Scatterbrain: Unifying Sparse and Low-rank Attention Approximation. CoRR (2021)
Beidi Chen, Tri Dao, Kaizhao Liang, Jiaming Yang, Zhao Song, Atri Rudra, Christopher Ré
Pixelated Butterfly: Simple and Efficient Sparse training for Neural Network Models. CoRR (2021)
Shabnam Daghaghi, Tharun Medini, Nicholas Meisburger, Beidi Chen, Mengnan Zhao, Anshumali Shrivastava
A Tale of Two Efficient and Informative Negative Sampling Distributions. ICML (2021)
Beidi Chen, Zichang Liu, Binghui Peng, Zhaozhuo Xu, Jonathan Lingjie Li, Tri Dao, Zhao Song, Anshumali Shrivastava, Christopher Ré
MONGOOSE: A Learnable LSH Framework for Efficient Neural Network Training. ICLR (2021)
Shabnam Daghaghi, Tharun Medini, Beidi Chen, Mengnan Zhao, Anshumali Shrivastava
A Constant-time Adaptive Negative Sampling. CoRR (2020)
Zichang Liu, Zhaozhuo Xu, Alan Baonan Ji, Jonathan Li, Beidi Chen, Anshumali Shrivastava
Climbing the WOL: Training for Cheaper Inference. CoRR (2020)
Beidi Chen, Weiyang Liu, Zhiding Yu, Jan Kautz, Anshumali Shrivastava, Animesh Garg, Animashree Anandkumar
Angular Visual Hardness. ICML (2020)
Tharun Medini, Beidi Chen, Anshumali Shrivastava
SOLAR: Sparse Orthogonal Learned and Random Embeddings. CoRR (2020)
Beidi Chen, Tharun Medini, James Farwell, Sameh Gobriel, Tsung-Yuan Charlie Tai, Anshumali Shrivastava
SLIDE : In Defense of Smart Algorithms over Hardware Acceleration for Large-Scale Deep Learning Systems. MLSys (2020)
M. Sadegh Riazi, Beidi Chen, Anshumali Shrivastava, Dan S. Wallach, Farinaz Koushanfar
Sub-Linear Privacy-Preserving Near-Neighbor Search. IACR Cryptol. ePrint Arch. 2019 (2019)
Beidi Chen, Yingchen Xu, Anshumali Shrivastava
Fast and Accurate Stochastic Gradient Estimation. NeurIPS (2019)
Beidi Chen, Yingchen Xu, Anshumali Shrivastava
Lsh-sampling Breaks the Computation Chicken-and-egg Loop in Adaptive Stochastic Gradient Estimation. CoRR (2019)
Beidi Chen, Tharun Medini, Anshumali Shrivastava
SLIDE : In Defense of Smart Algorithms over Hardware Acceleration for Large-Scale Deep Learning Systems. CoRR (2019)
Beidi Chen, Weiyang Liu, Animesh Garg, Zhiding Yu, Anshumali Shrivastava, Jan Kautz, Anima Anandkumar
Angular Visual Hardness. CoRR (2019)
Beidi Chen, Anshumali Shrivastava
Densified Winner Take All (WTA) Hashing for Sparse Datasets. UAI (2018)
Beidi Chen, Yingchen Xu, Anshumali Shrivastava
Lsh-Sampling breaks the Computational chicken-and-egg Loop in adaptive stochastic Gradient estimation. ICLR (Workshop) (2018)
Beidi Chen, Anshumali Shrivastava, Rebecca C. Steorts
Unique Entity Estimation with Application to the Syrian Conflict. CoRR (2017)
M. Sadegh Riazi, Beidi Chen, Anshumali Shrivastava, Dan S. Wallach, Farinaz Koushanfar
Sub-linear Privacy-preserving Search with Untrusted Server and Semi-honest Parties. CoRR (2016)
Beidi Chen, Anshumali Shrivastava
Revisiting Winner Take All (WTA) Hashing for Sparse Datasets. CoRR (2016)