Kaifeng Lyu

Publication Activity (10 Years)

Years Active: 2018-2024
Publications (10 Years): 37

Top Topics

Multilayer Neural Networks

Matrix Factorization

Delta Bar Delta

Top Venues

Publications

Vedant Shah, Dingli Yu, Kaifeng Lyu, Simon Park, Nan Rosemary Ke, Michael Mozer, Yoshua Bengio, Sanjeev Arora, Anirudh Goyal
AI-Assisted Generation of Difficult Math Questions. CoRR (2024)
Yongchao Zhou, Kaifeng Lyu, Ankit Singh Rawat, Aditya Krishna Menon, Afshin Rostamizadeh, Sanjiv Kumar, Jean-François Kagy, Rishabh Agarwal
DistillSpec: Improving Speculative Decoding via Knowledge Distillation. ICLR (2024)
Kaifeng Lyu, Jikai Jin, Zhiyuan Li, Simon Shaolei Du, Jason D. Lee, Wei Hu
Dichotomy of Early and Late Phase Implicit Biases Can Provably Induce Grokking. ICLR (2024)
Abhishek Panigrahi, Nikunj Saunshi, Kaifeng Lyu, Sobhan Miryoosefi, Sashank J. Reddi, Satyen Kale, Sanjiv Kumar
Efficient Stagewise Pretraining via Progressive Subnetworks. CoRR (2024)
Xinran Gu, Kaifeng Lyu, Sanjeev Arora, Jingzhao Zhang, Longbo Huang
A Quadratic Synchronization Rule for Distributed Deep Learning. ICLR (2024)
Kaiyue Wen, Xingyu Dang, Kaifeng Lyu
RNNs are not Transformers (Yet): The Key Bottleneck on In-context Retrieval. CoRR (2024)
Xiangyu Qi, Ashwinee Panda, Kaifeng Lyu, Xiao Ma, Subhrajit Roy, Ahmad Beirami, Prateek Mittal, Peter Henderson
Safety Alignment Should Be Made More Than Just a Few Tokens Deep. CoRR (2024)
Runzhe Wang, Sadhika Malladi, Tianhao Wang, Kaifeng Lyu, Zhiyuan Li
The Marginal Value of Momentum for Small Learning Rate SGD. ICLR (2024)
Kaifeng Lyu, Haoyu Zhao, Xinran Gu, Dingli Yu, Anirudh Goyal, Sanjeev Arora
Keeping LLMs Aligned After Fine-tuning: The Crucial Role of Prompt Templates. CoRR (2024)
Xinran Gu, Kaifeng Lyu, Longbo Huang, Sanjeev Arora
Why (and When) does Local SGD Generalize Better than SGD? CoRR (2023)
Jikai Jin, Zhiyuan Li, Kaifeng Lyu, Simon S. Du, Jason D. Lee
Understanding Incremental Learning of Gradient Descent: A Fine-grained Analysis of Matrix Sensing. CoRR (2023)
Yongchao Zhou, Kaifeng Lyu, Ankit Singh Rawat, Aditya Krishna Menon, Afshin Rostamizadeh, Sanjiv Kumar, Jean-François Kagy, Rishabh Agarwal
DistillSpec: Improving Speculative Decoding via Knowledge Distillation. CoRR (2023)
Xinran Gu, Kaifeng Lyu, Sanjeev Arora, Jingzhao Zhang, Longbo Huang
A Quadratic Synchronization Rule for Distributed Deep Learning. CoRR (2023)
Jikai Jin, Zhiyuan Li, Kaifeng Lyu, Simon Shaolei Du, Jason D. Lee
Understanding Incremental Learning of Gradient Descent: A Fine-grained Analysis of Matrix Sensing. ICML (2023)
Kaifeng Lyu, Jikai Jin, Zhiyuan Li, Simon S. Du, Jason D. Lee, Wei Hu
Dichotomy of Early and Late Phase Implicit Biases Can Provably Induce Grokking. CoRR (2023)
Runzhe Wang, Sadhika Malladi, Tianhao Wang, Kaifeng Lyu, Zhiyuan Li
The Marginal Value of Momentum for Small Learning Rate SGD. CoRR (2023)
Xinran Gu, Kaifeng Lyu, Longbo Huang, Sanjeev Arora
Why (and When) does Local SGD Generalize Better than SGD? ICLR (2023)
Sadhika Malladi, Kaifeng Lyu, Abhishek Panigrahi, Sanjeev Arora
On the SDEs and Scaling Rules for Adaptive Gradient Algorithms. NeurIPS (2022)
Kaifeng Lyu, Zhiyuan Li, Sanjeev Arora
Understanding the Generalization Benefit of Normalization Layers: Sharpness Reduction. CoRR (2022)
Arushi Gupta, Nikunj Saunshi, Dingli Yu, Kaifeng Lyu, Sanjeev Arora
New Definitions and Evaluations for Saliency Methods: Staying Intrinsic, Complete and Sound. CoRR (2022)
Arushi Gupta, Nikunj Saunshi, Dingli Yu, Kaifeng Lyu, Sanjeev Arora
New Definitions and Evaluations for Saliency Methods: Staying Intrinsic, Complete and Sound. NeurIPS (2022)
Sadhika Malladi, Kaifeng Lyu, Abhishek Panigrahi, Sanjeev Arora
On the SDEs and Scaling Rules for Adaptive Gradient Algorithms. CoRR (2022)
Kaifeng Lyu, Zhiyuan Li, Sanjeev Arora
Understanding the Generalization Benefit of Normalization Layers: Sharpness Reduction. NeurIPS (2022)
Kaifeng Lyu, Zhiyuan Li, Runzhe Wang, Sanjeev Arora
Gradient Descent on Two-layer Nets: Margin Maximization and Simplicity Bias. NeurIPS (2021)
Zhiyuan Li, Yuping Luo, Kaifeng Lyu
Towards Resolving the Implicit Bias of Gradient Descent for Matrix Factorization: Greedy Low-Rank Learning. ICLR (2021)
Kaifeng Lyu, Zhiyuan Li, Runzhe Wang, Sanjeev Arora
Gradient Descent on Two-layer Nets: Margin Maximization and Simplicity Bias. CoRR (2021)
Zhiyuan Li, Kaifeng Lyu, Sanjeev Arora
Reconciling Modern Deep Learning with Traditional Optimization Analyses: The Intrinsic Learning Rate. NeurIPS (2020)
Zhiyuan Li, Kaifeng Lyu, Sanjeev Arora
Reconciling Modern Deep Learning with Traditional Optimization Analyses: The Intrinsic Learning Rate. CoRR (2020)
Kaifeng Lyu, Jian Li
Gradient Descent Maximizes the Margin of Homogeneous Neural Networks. ICLR (2020)
Zhiyuan Li, Yuping Luo, Kaifeng Lyu
Towards Resolving the Implicit Bias of Gradient Descent for Matrix Factorization: Greedy Low-Rank Learning. CoRR (2020)
Sanjeev Arora, Zhiyuan Li, Kaifeng Lyu
Theoretical Analysis of Auto Rate-Tuning by Batch Normalization. ICLR (Poster) (2019)
Kaifeng Lyu, Jian Li
Gradient Descent Maximizes the Margin of Homogeneous Neural Networks. CoRR (2019)
Lijie Chen, Shafi Goldwasser, Kaifeng Lyu, Guy N. Rothblum, Aviad Rubinstein
Fine-grained Complexity Meets IP = PSPACE. SODA (2019)
Lijie Chen, Shafi Goldwasser, Kaifeng Lyu, Guy N. Rothblum, Aviad Rubinstein
Fine-grained Complexity Meets IP = PSPACE. CoRR (2018)
Ran Duan, Kaifeng Lyu, Yuanhang Xie
Single-Source Bottleneck Path Algorithm Faster than Sorting for Sparse Graphs. ICALP (2018)
Sanjeev Arora, Zhiyuan Li, Kaifeng Lyu
Theoretical Analysis of Auto Rate-Tuning by Batch Normalization. CoRR (2018)
Ran Duan, Kaifeng Lyu, Hongxun Wu, Yuanhang Xie
Single-Source Bottleneck Path Algorithm Faster than Sorting for Sparse Graphs. CoRR (2018)