Zhiyuan Li

Publication Activity (10 Years)

Years Active: 2016-2024
Publications (10 Years): 55

Top Topics

Regularization Methods

Stochastic Gradient Descent

Delta Bar Delta

Top Venues

Publications

Hong Liu, Zhiyuan Li, David Leo Wright Hall, Percy Liang, Tengyu Ma
Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-training. ICLR (2024)
Kaifeng Lyu, Jikai Jin, Zhiyuan Li, Simon Shaolei Du, Jason D. Lee, Wei Hu
Dichotomy of Early and Late Phase Implicit Biases Can Provably Induce Grokking. ICLR (2024)
Runzhe Wang, Sadhika Malladi, Tianhao Wang, Kaifeng Lyu, Zhiyuan Li
The Marginal Value of Momentum for Small Learning Rate SGD. ICLR (2024)
Kaiyue Wen, Zhiyuan Li, Tengyu Ma
Sharpness Minimization Algorithms Do Not Only Minimize Sharpness To Achieve Better Generalization. NeurIPS (2023)
Jikai Jin, Zhiyuan Li, Kaifeng Lyu, Simon S. Du, Jason D. Lee
Understanding Incremental Learning of Gradient Descent: A Fine-grained Analysis of Matrix Sensing. CoRR (2023)
Jikai Jin, Zhiyuan Li, Kaifeng Lyu, Simon Shaolei Du, Jason D. Lee
Understanding Incremental Learning of Gradient Descent: A Fine-grained Analysis of Matrix Sensing. ICML (2023)
Kaifeng Lyu, Jikai Jin, Zhiyuan Li, Simon S. Du, Jason D. Lee, Wei Hu
Dichotomy of Early and Late Phase Implicit Biases Can Provably Induce Grokking. CoRR (2023)
Runzhe Wang, Sadhika Malladi, Tianhao Wang, Kaifeng Lyu, Zhiyuan Li
The Marginal Value of Momentum for Small Learning Rate SGD. CoRR (2023)
Zhiyuan Li, Tianhao Wang, Dingli Yu
Fast Mixing of Stochastic Gradient Descent with Normalization and Weight Decay. NeurIPS (2022)
Kaifeng Lyu, Zhiyuan Li, Sanjeev Arora
Understanding the Generalization Benefit of Normalization Layers: Sharpness Reduction. CoRR (2022)
Sanjeev Arora, Zhiyuan Li, Abhishek Panigrahi
Understanding Gradient Descent on Edge of Stability in Deep Learning. CoRR (2022)
Zhiyuan Li, Tianhao Wang, Sanjeev Arora
What Happens after SGD Reaches Zero Loss? --A Mathematical Framework. ICLR (2022)
Zhiyuan Li, Srinadh Bhojanapalli, Manzil Zaheer, Sashank J. Reddi, Sanjiv Kumar
Robust Training of Neural Networks using Scale Invariant Architectures. CoRR (2022)
Zhiyuan Li, Srinadh Bhojanapalli, Manzil Zaheer, Sashank J. Reddi, Sanjiv Kumar
Robust Training of Neural Networks Using Scale Invariant Architectures. ICML (2022)
Kaifeng Lyu, Zhiyuan Li, Sanjeev Arora
Understanding the Generalization Benefit of Normalization Layers: Sharpness Reduction. NeurIPS (2022)
Sanjeev Arora, Zhiyuan Li, Abhishek Panigrahi
Understanding Gradient Descent on the Edge of Stability in Deep Learning. ICML (2022)
Zhiyuan Li, Tianhao Wang, Jason D. Lee, Sanjeev Arora
Implicit Bias of Gradient Descent on Reparametrized Models: On Equivalence to Mirror Descent. CoRR (2022)
Zhiyuan Li, Tianhao Wang, Jason D. Lee, Sanjeev Arora
Implicit Bias of Gradient Descent on Reparametrized Models: On Equivalence to Mirror Descent. NeurIPS (2022)
Kaifeng Lyu, Zhiyuan Li, Runzhe Wang, Sanjeev Arora
Gradient Descent on Two-layer Nets: Margin Maximization and Simplicity Bias. NeurIPS (2021)
Zhiyuan Li, Tianhao Wang, Sanjeev Arora
What Happens after SGD Reaches Zero Loss? -A Mathematical Framework. CoRR (2021)
Zhiyuan Li, Yi Zhang, Sanjeev Arora
Why Are Convolutional Nets More Sample-Efficient than Fully-Connected Nets? ICLR (2021)
Zhiyuan Li, Yuping Luo, Kaifeng Lyu
Towards Resolving the Implicit Bias of Gradient Descent for Matrix Factorization: Greedy Low-Rank Learning. ICLR (2021)
Simon S. Du, Wei Hu, Zhiyuan Li, Ruoqi Shen, Zhao Song, Jiajun Wu
When is particle filtering efficient for planning in partially observed linear dynamical systems? UAI (2021)
Zhiyuan Li, Sadhika Malladi, Sanjeev Arora
On the Validity of Modeling SGD with Stochastic Differential Equations (SDEs). CoRR (2021)
Zhiyuan Li, Sadhika Malladi, Sanjeev Arora
On the Validity of Modeling SGD with Stochastic Differential Equations (SDEs). NeurIPS (2021)
Kaifeng Lyu, Zhiyuan Li, Runzhe Wang, Sanjeev Arora
Gradient Descent on Two-layer Nets: Margin Maximization and Simplicity Bias. CoRR (2021)
Zhiyuan Li, Yi Zhang, Sanjeev Arora
Why Are Convolutional Nets More Sample-Efficient than Fully-Connected Nets? CoRR (2020)
Zhiyuan Li, Sanjeev Arora
An Exponential Learning Rate Schedule for Deep Learning. ICLR (2020)
Zhiyuan Li, Kaifeng Lyu, Sanjeev Arora
Reconciling Modern Deep Learning with Traditional Optimization Analyses: The Intrinsic Learning Rate. NeurIPS (2020)
Zhiyuan Li, Kaifeng Lyu, Sanjeev Arora
Reconciling Modern Deep Learning with Traditional Optimization Analyses: The Intrinsic Learning Rate. CoRR (2020)
Simon S. Du, Wei Hu, Zhiyuan Li, Ruoqi Shen, Zhao Song, Jiajun Wu
When is Particle Filtering Efficient for POMDP Sequential Planning? CoRR (2020)
Sanjeev Arora, Simon S. Du, Zhiyuan Li, Ruslan Salakhutdinov, Ruosong Wang, Dingli Yu
Harnessing the Power of Infinitely Wide Deep Nets on Small-data Tasks. ICLR (2020)
Zhiyuan Li, Yuping Luo, Kaifeng Lyu
Towards Resolving the Implicit Bias of Gradient Descent for Matrix Factorization: Greedy Low-Rank Learning. CoRR (2020)
Wei Hu, Zhiyuan Li, Dingli Yu
Simple and Effective Regularization Methods for Training on Noisily Labeled Data with Generalization Guarantee. ICLR (2020)
Sanjeev Arora, Simon S. Du, Wei Hu, Zhiyuan Li, Ruslan Salakhutdinov, Ruosong Wang
On Exact Computation with an Infinitely Wide Neural Net. CoRR (2019)
Sanjeev Arora, Zhiyuan Li, Kaifeng Lyu
Theoretical Analysis of Auto Rate-Tuning by Batch Normalization. ICLR (Poster) (2019)
Wei Hu, Zhiyuan Li, Dingli Yu
Understanding Generalization of Deep Neural Networks Trained with Noisy Labels. CoRR (2019)
Sanjeev Arora, Simon S. Du, Wei Hu, Zhiyuan Li, Ruosong Wang
Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networks. ICML (2019)
Zhiyuan Li, Ruosong Wang, Dingli Yu, Simon S. Du, Wei Hu, Ruslan Salakhutdinov, Sanjeev Arora
Enhanced Convolutional Neural Tangent Kernels. CoRR (2019)
Sanjeev Arora, Simon S. Du, Wei Hu, Zhiyuan Li, Ruosong Wang
Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networks. CoRR (2019)
Sanjeev Arora, Simon S. Du, Zhiyuan Li, Ruslan Salakhutdinov, Ruosong Wang, Dingli Yu
Harnessing the Power of Infinitely Wide Deep Nets on Small-data Tasks. CoRR (2019)
Sanjeev Arora, Simon S. Du, Wei Hu, Zhiyuan Li, Ruslan Salakhutdinov, Ruosong Wang
On Exact Computation with an Infinitely Wide Neural Net. NeurIPS (2019)
Rohith Kuditipudi, Xiang Wang, Holden Lee, Yi Zhang, Zhiyuan Li, Wei Hu, Sanjeev Arora, Rong Ge
Explaining Landscape Connectivity of Low-cost Solutions for Multilayer Nets. CoRR (2019)
Rohith Kuditipudi, Xiang Wang, Holden Lee, Yi Zhang, Zhiyuan Li, Wei Hu, Rong Ge, Sanjeev Arora
Explaining Landscape Connectivity of Low-cost Solutions for Multilayer Nets. NeurIPS (2019)
Behnam Neyshabur, Zhiyuan Li, Srinadh Bhojanapalli, Yann LeCun, Nathan Srebro
The role of over-parametrization in generalization of neural networks. ICLR (Poster) (2019)
Zhiyuan Li, Sanjeev Arora
An Exponential Learning Rate Schedule for Deep Learning. CoRR (2019)
Elad Hazan, Wei Hu, Yuanzhi Li, Zhiyuan Li
Online Improper Learning with an Approximation Oracle. CoRR (2018)
Elad Hazan, Wei Hu, Yuanzhi Li, Zhiyuan Li
Online Improper Learning with an Approximation Oracle. NeurIPS (2018)
Behnam Neyshabur, Zhiyuan Li, Srinadh Bhojanapalli, Yann LeCun, Nathan Srebro
Towards Understanding the Role of Over-Parametrization in Generalization of Neural Networks. CoRR (2018)
Sanjeev Arora, Zhiyuan Li, Kaifeng Lyu
Theoretical Analysis of Auto Rate-Tuning by Batch Normalization. CoRR (2018)
Zhiyuan Li, Yicheng Liu, Pingzhong Tang, Tingting Xu, Wei Zhan
Stability of Generalized Two-sided Markets with Transaction Thresholds. AAMAS (2017)
Dylan J. Foster, Zhiyuan Li, Thodoris Lykouris, Karthik Sridharan, Éva Tardos
Learning in Games: Robustness of Fast Convergence. NIPS (2016)
Yexiang Xue, Zhiyuan Li, Stefano Ermon, Carla P. Gomes, Bart Selman
Solving Marginal MAP Problems with NP Oracles and Parity Constraints. CoRR (2016)
Dylan J. Foster, Zhiyuan Li, Thodoris Lykouris, Karthik Sridharan, Éva Tardos
Fast Convergence of Common Learning Algorithms in Games. CoRR (2016)
Yexiang Xue, Zhiyuan Li, Stefano Ermon, Carla P. Gomes, Bart Selman
Solving Marginal MAP Problems with NP Oracles and Parity Constraints. NIPS (2016)