Ruoyu Sun

Publication Activity (10 Years)

Years Active: 2011-2024
Publications (10 Years): 77

Top Topics

Rademacher Complexity

Top Venues

Publications

Bohan Wang, Yushun Zhang, Huishuai Zhang, Qi Meng, Ruoyu Sun, Zhi-Ming Ma, Tie-Yan Liu, Zhi-Quan Luo, Wei Chen
Provable Adaptivity of Adam under Non-uniform Smoothness. KDD (2024)
Yupeng Chen, Senmiao Wang, Zhihang Lin, Zeyu Qin, Yushun Zhang, Tian Ding, Ruoyu Sun
MoFO: Momentum-Filtered Optimizer for Mitigating Forgetting in LLM Fine-Tuning. CoRR (2024)
Jiancong Xiao, Ruoyu Sun, Qi Long, Weijie J. Su
Bridging the Gap: Rademacher Complexity in Robust and Standard Generalization. CoRR (2024)
Yushun Zhang, Congliang Chen, Tian Ding, Ziniu Li, Ruoyu Sun, Zhi-Quan Luo
Why Transformers Need Adam: A Hessian Perspective. CoRR (2024)
Yushun Zhang, Congliang Chen, Ziniu Li, Tian Ding, Chenwei Wu, Yinyu Ye, Zhi-Quan Luo, Ruoyu Sun
Adam-mini: Use Fewer Learning Rates To Gain More. CoRR (2024)
Yite Wang, Jiahao Su, Hanlin Lu, Cong Xie, Tianyi Liu, Jianbo Yuan, Haibin Lin, Ruoyu Sun, Hongxia Yang
LEMON: Lossless model expansion. ICLR (2024)
Jiancong Xiao, Ruoyu Sun, Qi Long, Weijie Su
Bridging the Gap: Rademacher Complexity in Robust and Standard Generalization. COLT (2024)
Huang Huang, Fei Yu, Jianqing Zhu, Xuening Sun, Hao Cheng, Dingjie Song, Zhihong Chen, Mosen Alharthi, Bang An, Juncai He, Ziche Liu, Junying Chen, Jianquan Li, Benyou Wang, Lian Zhang, Ruoyu Sun, Xiang Wan, Haizhou Li, Jinchao Xu
AceGPT, Localizing Large Language Models in Arabic. NAACL-HLT (2024)
Bohan Wang, Huishuai Zhang, Qi Meng, Ruoyu Sun, Zhi-Ming Ma, Wei Chen
On the Convergence of Adam under Non-uniform Smoothness: Separability from SGDM and Beyond. CoRR (2024)
Bingheng Li, Linxin Yang, Yupeng Chen, Senmiao Wang, Qian Chen, Haitao Mao, Yao Ma, Akang Wang, Tian Ding, Jiliang Tang, Ruoyu Sun
PDHG-Unrolled Learning-to-Optimize Method for Large-Scale Linear Programming. CoRR (2024)
Qingyu Han, Linxin Yang, Qian Chen, Xiang Zhou, Dong Zhang, Akang Wang, Ruoyu Sun, Xiaodong Luo
A GNN-Guided Predict-and-Search Framework for Mixed-Integer Linear Programming. ICLR (2023)
Jiancong Xiao, Ruoyu Sun, Zhi-Quan Luo
PAC-Bayesian Spectrally-Normalized Bounds for Adversarially Robust Generalization. NeurIPS (2023)
Huang Huang, Fei Yu, Jianqing Zhu, Xuening Sun, Hao Cheng, Dingjie Song, Zhihong Chen, Abdulmohsen Alharthi, Bang An, Ziche Liu, Zhiyi Zhang, Junying Chen, Jianquan Li, Benyou Wang, Lian Zhang, Ruoyu Sun, Xiang Wan, Haizhou Li, Jinchao Xu
AceGPT, Localizing Large Language Models in Arabic. CoRR (2023)
Yite Wang, Jing Wu, Naira Hovakimyan, Ruoyu Sun
Balanced Training for Sparse GANs. NeurIPS (2023)
Yite Wang, Dawei Li, Ruoyu Sun
NTK-SAP: Improving neural network pruning by aligning training dynamics. CoRR (2023)
Jiancong Xiao, Ruoyu Sun, Zhi-Quan Luo
PAC-Bayesian Spectrally-Normalized Bounds for Adversarially Robust Generalization. CoRR (2023)
Yite Wang, Dawei Li, Ruoyu Sun
NTK-SAP: Improving neural network pruning by aligning training dynamics. ICLR (2023)
Dmitry Rybin, Ruoyu Sun, Zhi-Quan Luo
Invariant Layers for Graphs with Nodes of Different Types. CoRR (2023)
Chenxiao Yang, Qitian Wu, David Wipf, Ruoyu Sun, Junchi Yan
How Graph Neural Networks Learn: Lessons from Training Dynamics in Function Space. CoRR (2023)
Ziniu Li, Tian Xu, Yushun Zhang, Yang Yu, Ruoyu Sun, Zhi-Quan Luo
ReMax: A Simple, Effective, and Efficient Reinforcement Learning Method for Aligning Large Language Models. CoRR (2023)
Feng Xiao, Ruoyu Sun, Jicong Fan
Restricted Generative Projection for One-Class Classification and Anomaly Detection. CoRR (2023)
Yite Wang, Jiahao Su, Hanlin Lu, Cong Xie, Tianyi Liu, Jianbo Yuan, Haibin Lin, Ruoyu Sun, Hongxia Yang
LEMON: Lossless model expansion. CoRR (2023)
Yite Wang, Jing Wu, Naira Hovakimyan, Ruoyu Sun
Double Dynamic Sparse Training for GANs. CoRR (2023)
Tian Ding, Dawei Li, Ruoyu Sun
Suboptimal Local Minima Exist for Wide Neural Networks with Smooth Activations. Math. Oper. Res. 47 (4) (2022)
Jiawei Zhang, Yushun Zhang, Mingyi Hong, Ruoyu Sun, Zhi-Quan Luo
When Expressivity Meets Trainability: Fewer than n Neurons Can Work. CoRR (2022)
Haoxiang Wang, Yite Wang, Ruoyu Sun, Bo Li
Global Convergence of MAML and Theory-Inspired Neural Architecture Search for Few-Shot Learning. CoRR (2022)
Yushun Zhang, Congliang Chen, Naichen Shi, Ruoyu Sun, Zhi-Quan Luo
Adam Can Converge Without Any Modification On Update Rules. NeurIPS (2022)
Jiancong Xiao, Yanbo Fan, Ruoyu Sun, Jue Wang, Zhi-Quan Luo
Stability Analysis and Generalization Bounds of Adversarial Training. CoRR (2022)
Haoxiang Wang, Yite Wang, Ruoyu Sun, Bo Li
Global Convergence of MAML and Theory-Inspired Neural Architecture Search for Few-Shot Learning. CVPR (2022)
Jiancong Xiao, Yanbo Fan, Ruoyu Sun, Jue Wang, Zhi-Quan Luo
Stability Analysis and Generalization Bounds of Adversarial Training. NeurIPS (2022)
Dachao Lin, Ruoyu Sun, Zhihua Zhang
On the landscape of one-hidden-layer sparse networks and beyond. Artif. Intell. 309 (2022)
Shiyu Liang, Ruoyu Sun, R. Srikant
Revisiting Landscape Analysis in Deep Neural Networks: Eliminating Decreasing Paths to Infinity. SIAM J. Optim. 32 (4) (2022)
Yushun Zhang, Congliang Chen, Naichen Shi, Ruoyu Sun, Zhi-Quan Luo
Adam Can Converge Without Any Modification on Update Rules. CoRR (2022)
Dawei Li, Tian Ding, Ruoyu Sun
On the Benefit of Width for Neural Networks: Disappearance of Basins. SIAM J. Optim. 32 (3) (2022)
Jiancong Xiao, Yanbo Fan, Ruoyu Sun, Zhi-Quan Luo
Adversarial Rademacher Complexity of Deep Neural Networks. CoRR (2022)
Tiantian Fang, Ruoyu Sun, Alexander G. Schwing
DigGAN: Discriminator gradIent Gap Regularization for GAN Training with Limited Data. NeurIPS (2022)
Tiantian Fang, Ruoyu Sun, Alexander G. Schwing
DigGAN: Discriminator gradIent Gap Regularization for GAN Training with Limited Data. CoRR (2022)
Bohan Wang, Qi Meng, Huishuai Zhang, Ruoyu Sun, Wei Chen, Zhi-Ming Ma, Tie-Yan Liu
Does Momentum Change the Implicit Regularization on Separable Data? NeurIPS (2022)
Bohan Wang, Qi Meng, Huishuai Zhang, Ruoyu Sun, Wei Chen, Zhi-Ming Ma
Momentum Doesn't Change the Implicit Bias. CoRR (2021)
Peijun Xiao, Zhisheng Xiao, Ruoyu Sun
Times Slower Than the Randomized Version. SIAM J. Optim. 31 (4) (2021)
Yinchen Shen, Zhiguo Wang, Ruoyu Sun, Xiaojing Shen
Towards Understanding the Impact of Model Size on Differential Private Classification. CoRR (2021)
Akhil Gupta, Lavanya Marla, Ruoyu Sun, Naman Shukla, Arinbjörn Kolbeinsson
PenDer: Incorporating Shape Constraints via Penalized Derivatives. AAAI (2021)
Jiawei Zhang, Yushun Zhang, Mingyi Hong, Ruoyu Sun, Zhi-Quan Luo
When Expressivity Meets Trainability: Fewer than $n$ Neurons Can Work. NeurIPS (2021)
Shiyu Liang, Ruoyu Sun, R. Srikant
Achieving Small Test Error in Mildly Overparameterized Neural Networks. CoRR (2021)
Dawei Li, Ruoyu Sun
On a Faster R-Linear Convergence Rate of the Barzilai-Borwein Method. CoRR (2021)
Zhiguo Wang, Xintong Wang, Ruoyu Sun, Tsung-Hui Chang
Federated Semi-Supervised Learning with Class Distribution Mismatch. CoRR (2021)
Dachao Lin, Ruoyu Sun, Zhihua Zhang
Faster Directional Convergence of Linear Neural Networks under Spherically Symmetric Data. NeurIPS (2021)
Naichen Shi, Dawei Li, Mingyi Hong, Ruoyu Sun
RMSprop converges with proper hyper-parameter. ICLR (2021)
Yahang Li, Zepeng Wang, Ruoyu Sun, Fan Lam
H-MRSI Using Learned Component-Specific Representations. IEEE Trans. Medical Imaging 40 (4) (2021)
Ruoyu Sun, Yinyu Ye
gap with randomized version. Math. Program. 185 (1-2) (2021)
Tian Ye, Peijun Xiao, Ruoyu Sun
DEED: A General Quantization Scheme for Communication Efficiency in Bits. CoRR (2020)
Ruoyu Sun, Dawei Li, Shiyu Liang, Tian Ding, Rayadurgam Srikant
The Global Landscape of Neural Networks: An Overview. IEEE Signal Process. Mag. 37 (5) (2020)
Jiawei Zhang, Peijun Xiao, Ruoyu Sun, Zhi-Quan Luo
A Single-Loop Smoothed Gradient Descent-Ascent Algorithm for Nonconvex-Concave Min-Max Problems. CoRR (2020)
Ruoyu Sun, Zhi-Quan Luo, Yinyu Ye
On the Efficiency of Random Permutation for ADMM and Coordinate Descent. Math. Oper. Res. 45 (1) (2020)
Ruoyu Sun, Tiantian Fang, Alexander G. Schwing
Towards a Better Global Loss Landscape of GANs. NeurIPS (2020)
Dachao Lin, Ruoyu Sun, Zhihua Zhang
Landscape of Sparse Linear Network: A Brief Investigation. CoRR (2020)
Ruoyu Sun, Dawei Li, Shiyu Liang, Tian Ding, R. Srikant
The Global Landscape of Neural Networks: An Overview. CoRR (2020)
Jiawei Zhang, Peijun Xiao, Ruoyu Sun, Zhi-Quan Luo
A Single-Loop Smoothed Gradient Descent-Ascent Algorithm for Nonconvex-Concave Min-Max Problems. NeurIPS (2020)
Ruoyu Sun, Tiantian Fang, Alexander G. Schwing
Towards a Better Global Loss Landscape of GANs. CoRR (2020)
Haoxiang Wang, Ruoyu Sun, Bo Li
Global Convergence and Induced Kernels of Gradient-Based Meta-Learning with Neural Nets. CoRR (2020)
Peijun Xiao, Zhisheng Xiao, Ruoyu Sun
Understanding Limitation of Two Symmetrized Orders by Worst-case Complexity. CoRR (2019)
Ishan Deshpande, Yuan-Ting Hu, Ruoyu Sun, Ayis Pyrros, Nasir Siddiqui, Sanmi Koyejo, Zhizhen Zhao, David A. Forsyth, Alexander G. Schwing
Max-Sliced Wasserstein Distance and Its Use for GANs. CVPR (2019)
Ishan Deshpande, Yuan-Ting Hu, Ruoyu Sun, Ayis Pyrros, Nasir Siddiqui, Sanmi Koyejo, Zhizhen Zhao, David A. Forsyth, Alexander G. Schwing
Max-Sliced Wasserstein Distance and its use for GANs. CoRR (2019)
Tian Ding, Dawei Li, Ruoyu Sun
Sub-Optimal Local Minima Exist for Almost All Over-parameterized Neural Networks. CoRR (2019)
Ruoyu Sun
Optimization for deep learning: theory and algorithms. CoRR (2019)
Xiangyi Chen, Sijia Liu, Ruoyu Sun, Mingyi Hong
On the Convergence of A Class of Adam-Type Algorithms for Non-Convex Optimization. ICLR (Poster) (2019)
Wei Liu, Ruoyu Sun, Zhi-Quan Luo
Globally Optimal Joint Uplink Base Station Association and Beamforming. IEEE Trans. Commun. 67 (9) (2019)
Shiyu Liang, Ruoyu Sun, R. Srikant
Revisiting Landscape Analysis in Deep Neural Networks: Eliminating Decreasing Paths to Infinity. CoRR (2019)
Shiyu Liang, Ruoyu Sun, Yixuan Li, R. Srikant
Understanding the Loss Surface of Neural Networks for Binary Classification. CoRR (2018)
Shiyu Liang, Ruoyu Sun, Jason D. Lee, R. Srikant
Adding One Neuron Can Eliminate All Bad Local Minima. NeurIPS (2018)
Shiyu Liang, Ruoyu Sun, Jason D. Lee, R. Srikant
Adding One Neuron Can Eliminate All Bad Local Minima. CoRR (2018)
Xiangyi Chen, Sijia Liu, Ruoyu Sun, Mingyi Hong
On the Convergence of A Class of Adam-Type Algorithms for Non-Convex Optimization. CoRR (2018)
Shiyu Liang, Ruoyu Sun, Yixuan Li, Rayadurgam Srikant
Understanding the Loss Surface of Neural Networks for Binary Classification. ICML (2018)
Shiyu Liang, Ruoyu Sun, Yixuan Li, R. Srikant
Understanding the Loss Surface of Single-Layered Neural Networks for Binary Classification. ICLR (Workshop) (2018)
Dawei Li, Tian Ding, Ruoyu Sun
Over-Parameterized Deep Neural Networks Have No Strict Local Minima For Any Continuous Activations. CoRR (2018)
Ruoyu Sun, Zhi-Quan Luo
Guaranteed Matrix Completion via Non-Convex Factorization. IEEE Trans. Inf. Theory 62 (11) (2016)
Ruoyu Sun, Yinyu Ye
Worst-case Complexity of Cyclic Coordinate Descent: $O(n^2)$ Gap with Randomized Version. CoRR (2016)
Ruoyu Sun, Zhi-Quan Luo
Guaranteed Matrix Completion via Nonconvex Factorization. FOCS (2015)
Ruoyu Sun, Zhi-Quan Luo
Interference Alignment Using Finite and Dependent Channel Extensions: The Single Beam Case. IEEE Trans. Inf. Theory 61 (1) (2015)
Wei Liu, Ruoyu Sun, Zhi-Quan Luo, Jiandong Li
Globally Optimal Joint Uplink Base Station Association and Beamforming. CoRR (2015)
Aritra Konar, Ruoyu Sun, Nicholas D. Sidiropoulos, Zhi-Quan Luo
Interference alignment via Feasible Point Pursuit. SPAWC (2015)
Ruoyu Sun, Mingyi Hong
Improved Iteration Complexity Bounds of Cyclic Block Coordinate Descent for Convex Problems. NIPS (2015)
Ruoyu Sun, Mingyi Hong, Zhi-Quan Luo
Joint Downlink Base Station Association and Power Control for Max-Min Fairness: Computation and Complexity. IEEE J. Sel. Areas Commun. 33 (6) (2015)
Ruoyu Sun, Zhi-Quan Luo
Guaranteed Matrix Completion via Non-convex Factorization. CoRR (2014)
Ruoyu Sun, Mingyi Hong, Zhi-Quan Luo
Joint Downlink Base Station Association and Power Control for Max-Min Fairness: Computation and Complexity. CoRR (2014)
Hadi Baligh, Mingyi Hong, Wei-Cheng Liao, Zhi-Quan Luo, Meisam Razaviyayn, Maziar Sanjabi, Ruoyu Sun
Cross-Layer Provision of Future Cellular Networks: A WMMSE-based approach. IEEE Signal Process. Mag. 31 (6) (2014)
Ruoyu Sun, Zhi-Quan Luo
Globally optimal joint uplink base station association and power control for max-min fairness. ICASSP (2014)
Hadi Baligh, Mingyi Hong, Wei-Cheng Liao, Zhi-Quan Luo, Meisam Razaviyayn, Maziar Sanjabi, Ruoyu Sun
Cross Layer Provision of Future Cellular Networks. CoRR (2014)
Ruoyu Sun, Zhi-Quan Luo
Two Performance-limiting Factors for Interference Alignment: Channel Diversity Order and the Number of Data Streams Per User. CoRR (2013)
Ruoyu Sun, Hadi Baligh, Zhi-Quan Luo
Long-term transmit point association for coordinated multipoint transmission by stochastic optimization. SPAWC (2013)
Mingyi Hong, Ruoyu Sun, Hadi Baligh, Zhi-Quan Luo
Joint Base Station Clustering and Beamformer Design for Partial Coordinated Transmission in Heterogeneous Networks. IEEE J. Sel. Areas Commun. 31 (2) (2013)
Enbin Song, Qingjiang Shi, Maziar Sanjabi, Ruoyu Sun, Zhi-Quan Luo
Robust SINR-Constrained MISO Downlink Beamforming: When is Semidefinite Programming Relaxation Tight? EURASIP J. Wirel. Commun. Netw. 2012 (2012)
Mingyi Hong, Meisam Razaviyayn, Ruoyu Sun, Zhi-Quan Luo
Joint transceiver design and base station clustering for heterogeneous networks. ACSCC (2012)
Mingyi Hong, Ruoyu Sun, Hadi Baligh, Zhi-Quan Luo
Joint Base Station Clustering and Beamformer Design for Partial Coordinated Transmission in Heterogenous Networks CoRR (2012)
Ruoyu Sun, Mingyi Hong, Zhi-Quan Luo
Optimal joint base station assignment and power allocation in a cellular network. SPAWC (2012)
Enbin Song, Qingjiang Shi, Maziar Sanjabi, Ruoyu Sun, Zhi-Quan Luo
Robust SINR-constrained MISO downlink beamforming: When is semidefinite programming relaxation tight? ICASSP (2011)