Zhao Song

ORCID

Publication Activity (10 Years)

Years Active: 2012-2024
Publications (10 Years): 286

Top Venues

Publications

Jiuxiang Gu, Chenyang Li, Yingyu Liang, Zhenmei Shi, Zhao Song
Exploring the Frontiers of Softmax: Provable Optimization, Applications in Diffusion Model, and Beyond. CoRR (2024)
Jiuxiang Gu, Yingyu Liang, Zhenmei Shi, Zhao Song, Yufa Zhou
Differential Privacy of Cross-Attention with Provable Guarantee. CoRR (2024)
Yeqi Gao, Lianke Qin, Zhao Song, Yitan Wang
A Sublinear Adversarial Training Algorithm. ICLR (2024)
Jerry Yao-Chieh Hu, Maojiang Su, En-Jui Kuo, Zhao Song, Han Liu
Computational Limits of Low-Rank Adaptation (LoRA) for Transformer-Based Models. CoRR (2024)
Zhao Song, Junze Yin, Lichen Zhang
Solving Attention Kernel Regression Problem via Pre-conditioner. AISTATS (2024)
Jiuxiang Gu, Yingyu Liang, Zhenmei Shi, Zhao Song, Chiwun Yang
Toward Infinite-Long Prefix in Transformer. CoRR (2024)
Lianke Qin, Zhao Song, Ruizhe Zhang
A General Algorithm for Solving Rank-one Matrix Sensing. AISTATS (2024)
Zhao Song, Junze Yin, Lichen Zhang, Ruizhe Zhang
Fast Dynamic Sampling for Determinantal Point Processes. AISTATS (2024)
Jiuxiang Gu, Yingyu Liang, Zhenmei Shi, Zhao Song, Yufa Zhou
Unraveling the Smoothness Properties of Diffusion Models: A Gaussian Mixture Perspective. CoRR (2024)
Jiuxiang Gu, Chenyang Li, Yingyu Liang, Zhenmei Shi, Zhao Song, Tianyi Zhou
Fourier Circuits in Neural Networks: Unlocking the Potential of Large Language Models in Mathematical Reasoning and Modular Arithmetic. CoRR (2024)
Jiuxiang Gu, Yingyu Liang, Zhizhou Sha, Zhenmei Shi, Zhao Song
Differential Privacy Mechanisms in Neural Tangent Kernel Regression. CoRR (2024)
Timothy Chu, Zhao Song, Chiwun Yang
How to Protect Copyright Data in Optimization of Large Language Models? AAAI (2024)
Josh Alman, Zhao Song
How to Capture Higher-order Correlations? Generalizing Matrix Softmax Attention to Kronecker Computation. ICLR (2024)
Yuzhou Gu, Zhao Song, Junze Yin, Lichen Zhang
Low Rank Matrix Completion via Robust Alternating Minimization in Nearly Linear Time. ICLR (2024)
Haotian Jiang, Yin Tat Lee, Zhao Song, Lichen Zhang
) Time. SODA (2024)
Jerry Yao-Chieh Hu, Weimin Wu, Zhuoru Li, Zhao Song, Han Liu
On Statistical Rates and Provably Efficient Criteria of Latent Diffusion Transformers (DiTs). CoRR (2024)
Yeqi Gao, Zhao Song, Ruizhe Zhang
Quantum Speedup for Spectral Approximation of Kronecker Products. CoRR (2024)
Yeqi Gao, Yuzhou Gu, Zhao Song
Binary Hypothesis Testing for Softmax Models and Leverage Score Models. CoRR (2024)
Yichuan Deng, Zhao Song, Chiwun Yang
Attention is Naturally Sparse with Gaussian Distributed Input. CoRR (2024)
Josh Alman, Zhao Song
The Fine-Grained Complexity of Gradient Computation for Training Large Language Models. CoRR (2024)
Jiuxiang Gu, Yingyu Liang, Heshan Liu, Zhenmei Shi, Zhao Song, Junze Yin
Conv-Basis: A New Paradigm for Efficient Attention Inference and Gradient Computation in Transformers. CoRR (2024)
Zhao Song, Lichen Zhang, Ruizhe Zhang
Training Multi-Layer Over-Parametrized Neural Network in Subquadratic Time. ITCS (2024)
Yichuan Deng, Zhao Song, Chiwun Yang
Enhancing Stochastic Gradient Descent: A Unified Framework and Novel Acceleration Methods for Faster Convergence. CoRR (2024)
Zhihang Li, Zhao Song, Weixin Wang, Junze Yin, Zheng Yu
How to Inverting the Leverage Score Distribution? CoRR (2024)
Jiuxiang Gu, Yingyu Liang, Zhenmei Shi, Zhao Song, Yufa Zhou
Tensor Attention Training: Provably Efficient Learning of Higher-order Transformers. CoRR (2024)
Lianke Qin, Aravind Reddy, Zhao Song
Online Adaptive Mahalanobis Distance Estimation. CoRR (2023)
Yeqi Gao, Zhao Song, Junze Yin
An Iterative Algorithm for Rescaled Hyperbolic Functions Regression. CoRR (2023)
Zichang Liu, Jue Wang, Tri Dao, Tianyi Zhou, Binhang Yuan, Zhao Song, Anshumali Shrivastava, Ce Zhang, Yuandong Tian, Christopher Ré, Beidi Chen
Deja Vu: Contextual Sparsity for Efficient LLMs at Inference Time. ICML (2023)
Yichuan Deng, Zhao Song, Shenghao Xie, Chiwun Yang
Unmasking Transformers: A Theoretical Approach to Data Recovery via Attention Weights. CoRR (2023)
Zhenyu Zhang, Ying Sheng, Tianyi Zhou, Tianlong Chen, Lianmin Zheng, Ruisi Cai, Zhao Song, Yuandong Tian, Christopher Ré, Clark W. Barrett, Zhangyang Wang, Beidi Chen
O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models. CoRR (2023)
Yichuan Deng, Yeqi Gao, Zhao Song
Solving Tensor Low Cycle Rank Approximation. CoRR (2023)
Yichuan Deng, Zhao Song, Lichen Zhang, Ruizhe Zhang
Efficient Algorithm for Solving Hyperbolic Programs. CoRR (2023)
Jan van den Brand, Zhao Song, Tianyi Zhou
Algorithm and Hardness for Dynamic Attention Maintenance in Large Language Models. CoRR (2023)
Yichuan Deng, Michalis Mamakos, Zhao Song
Clustered Linear Contextual Bandits with Knapsacks. CoRR (2023)
Zhao Song, Chiwun Yang
An Automatic Learning Rate Schedule Algorithm for Achieving Faster Convergence and Steeper Descent. CoRR (2023)
Yichuan Deng, Zhao Song, Tianyi Zhou
Superiority of Softmax: Unveiling the Performance Edge Over Linear Attention. CoRR (2023)
Zhao Song, Tianyi Zhou
Faster Sinkhorn's Algorithm with Small Treewidth. CoRR (2023)
Zhaozhuo Xu, Zhao Song, Anshumali Shrivastava
A Tale of Two Efficient Value Iteration Algorithms for Solving Linear MDPs with Large Action Space. AISTATS (2023)
Yeqi Gao, Sridhar Mahadevan, Zhao Song
An Over-parameterized Exponential Regression. CoRR (2023)
Josh Alman, Zhao Song
Fast Attention Requires Bounded Entries. CoRR (2023)
Zhao Song, Baocheng Sun, Omri Weinstein, Ruizhe Zhang
Quartic Samples Suffice for Fourier Interpolation. FOCS (2023)
Chenyang Li, Zhao Song, Weixin Wang, Chiwun Yang
A Theoretical Insight into Attack and Defense of Gradient Leakage in Transformer. CoRR (2023)
Yuzhou Gu, Zhao Song, Junze Yin, Lichen Zhang
Low Rank Matrix Completion via Robust Alternating Minimization in Nearly Linear Time. CoRR (2023)
Yichuan Deng, Yeqi Gao, Zhao Song
Solving Tensor Low Cycle Rank Approximation. BigData (2023)
Zhao Song, Mingquan Ye, Junze Yin, Lichen Zhang
Guarantee. CoRR (2023)
Yichuan Deng, Zhao Song, Shenghao Xie
Convergence of Two-Layer Regression with Nonlinear Units. CoRR (2023)
Junda Wu, Tong Yu, Rui Wang, Zhao Song, Ruiyi Zhang, Handong Zhao, Chaochao Lu, Shuai Li, Ricardo Henao
InfoPrompt: Information-Theoretic Soft Prompt Tuning for Natural Language Understanding. CoRR (2023)
Lianke Qin, Zhao Song, Yitan Wang
Fast Submodular Function Maximization. CoRR (2023)
Junda Wu, Tong Yu, Rui Wang, Zhao Song, Ruiyi Zhang, Handong Zhao, Chaochao Lu, Shuai Li, Ricardo Henao
InfoPrompt: Information-Theoretic Soft Prompt Tuning for Natural Language Understanding. NeurIPS (2023)
Xiaoxiao Li, Zhao Song, Guangyi Zhang
Sparse Convolution for Approximate Sparse Instance. CoRR (2023)
Shuai Li, Zhao Song, Yu Xia, Tong Yu, Tianyi Zhou
The Closeness of In-Context Learning and Weight Shifting for Softmax Regression. CoRR (2023)
Zhao Song, Mingquan Ye
Efficient Asynchronize Stochastic Gradient Algorithm with Structured Data. CoRR (2023)
Xiang Chen, Zhao Song, Baocheng Sun, Junze Yin, Danyang Zhuo
Query Complexity of Active Learning for Function Family With Nearly Orthogonal Basis. CoRR (2023)
Lijie Chen, Gillat Kol, Dmitry Paramonov, Raghuvansh R. Saxena, Zhao Song, Huacheng Yu
Towards Multi-Pass Streaming Lower Bounds for Optimal Approximation of Max-Cut. SODA (2023)
Haotian Jiang, Yin Tat Lee, Zhao Song, Lichen Zhang
) Time. CoRR (2023)
Yeqi Gao, Zhao Song, Xin Yang
Differentially Private Attention Computation. CoRR (2023)
Zhihang Li, Zhao Song, Zifan Wang, Junze Yin
Local Convergence of Approximate Newton Method for Two Layer Nonlinear Regression. CoRR (2023)
Yeqi Gao, Zhao Song, Weixin Wang, Junze Yin
A Fast Optimization View: Reformulating Single Layer Attention in LLM Based on Tensor and SVM Trick, and Solving It in Matrix Multiplication Time. CoRR (2023)
Timothy Chu, Zhao Song, Chiwun Yang
Fine-tune Language Models to Approximate Unbiased In-context Learning. CoRR (2023)
Lianke Qin, Zhao Song, Lichen Zhang, Danyang Zhuo
An Online and Unified Algorithm for Projection Matrix Vector Multiplication with Application to Empirical Risk Minimization. AISTATS (2023)
Lianke Qin, Zhao Song, Ruizhe Zhang
A General Algorithm for Solving Rank-one Matrix Sensing. CoRR (2023)
Yichuan Deng, Zhao Song, Junze Yin
Faster Robust Tensor Power Method for Arbitrary Order. CoRR (2023)
Yichuan Deng, Zhihang Li, Zhao Song
Attention Scheme Inspired Softmax Regression. CoRR (2023)
Lianke Qin, Zhao Song, Baocheng Sun
Is Solving Graph Neural Tangent Kernel Equivalent to Training Graph Neural Network? CoRR (2023)
Josh Alman, Zhao Song
How to Capture Higher-order Correlations? Generalizing Matrix Softmax Attention to Kronecker Computation. CoRR (2023)
Zhao Song, Xin Yang, Yuanyuan Yang, Lichen Zhang
Sketching Meets Differential Privacy: Fast Algorithm for Dynamic Kronecker Projection Maintenance. ICML (2023)
Kai Wang, Zhao Song, Georgios Theocharous, Sridhar Mahadevan
Smoothed Online Combinatorial Optimization Using Imperfect Predictions. AAAI (2023)
Zhihang Li, Zhao Song, Tianyi Zhou
Solving Regularized Exp, Cosh and Sinh Regression Problems. CoRR (2023)
Lianke Qin, Saayan Mitra, Zhao Song, Yuanyuan Yang, Tianyi Zhou
Fast Heavy Inner Product Identification Between Weights and Inputs in Neural Network Training. CoRR (2023)
Yaonan Jin, Daogao Liu, Zhao Song
Super-resolution and Robust Sparse Continuous Fourier Transform in Any Constant Dimension: Nearly Linear Time and Sample Complexity. SODA (2023)
Zhao Song, Weixin Wang, Junze Yin
A Unified Scheme of ResNet and Softmax. CoRR (2023)
Zhao Song, Weixin Wang, Chenbo Yin
Fast and Efficient Matching Algorithm with Deadline Instances. CoRR (2023)
Lianke Qin, Aravind Reddy, Zhao Song
Online Adaptive Mahalanobis Distance Estimation. BigData (2023)
Yuzhou Gu, Zhao Song, Lichen Zhang
A Nearly-Linear Time Algorithm for Structured Support Vector Machines. CoRR (2023)
Zhao Song, Guangyi Xu, Junze Yin
The Expressibility of Polynomial based Attention Scheme. CoRR (2023)
Raghav Addanki, Chenyang Li, Zhao Song, Chiwun Yang
One Pass Streaming Algorithm for Super Long Token Attention Approximation in Sublinear Space. CoRR (2023)
Lianke Qin, Saayan Mitra, Zhao Song, Yuanyuan Yang, Tianyi Zhou
Fast Heavy Inner Product Identification Between Weights and Inputs in Neural Network Training. BigData (2023)
Yeqi Gao, Zhao Song, Junze Yin
GradientCoin: A Peer-to-Peer Decentralized Large Language Models. CoRR (2023)
Zhao Song, Junze Yin, Lichen Zhang
Solving Attention Kernel Regression Problem via Pre-conditioner. CoRR (2023)
Zhao Song, Mingquan Ye, Junze Yin, Lichen Zhang
Efficient Alternating Minimization with Applications to Weighted Low Rank Approximation. CoRR (2023)
Anshumali Shrivastava, Zhao Song, Zhaozhuo Xu
A Theoretical Analysis Of Nearest Neighbor Search On Approximate Near Neighbor Graph. CoRR (2023)
Yeqi Gao, Zhao Song, Shenghao Xie
In-Context Learning for Attention Scheme: from Single Softmax Regression to Multiple Softmax Regression via a Tensor Trick. CoRR (2023)
Song Bian, Zhao Song, Junze Yin
Federated Empirical Risk Minimization via Second-Order Method. CoRR (2023)
Timothy Chu, Zhao Song, Chiwun Yang
How to Protect Copyright Data in Optimization of Large Language Models? CoRR (2023)
S. Cliff Liu, Zhao Song, Hengjie Zhang, Lichen Zhang, Tianyi Zhou
Space-Efficient Interior Point Method, with Applications to Linear Programming and Maximum Weight Bipartite Matching. ICALP (2023)
Zichang Liu, Jue Wang, Tri Dao, Tianyi Zhou, Binhang Yuan, Zhao Song, Anshumali Shrivastava, Ce Zhang, Yuandong Tian, Christopher Ré, Beidi Chen
Deja Vu: Contextual Sparsity for Efficient LLMs at Inference Time. CoRR (2023)
Sudhanshu Chanpuriya, Ryan A. Rossi, Anup B. Rao, Tung Mai, Nedim Lipka, Zhao Song, Cameron Musco
Exact Representation of Sparse Networks with Symmetric Nonnegative Embeddings. NeurIPS (2023)
Zhao Song, Yitan Wang, Zheng Yu, Lichen Zhang
Sketching for First Order Method: Efficient Algorithm for Low-Bandwidth Channel and Vulnerability. ICML (2023)
Xiaoxiao Li, Zhao Song, Jiaming Yang
Federated Adversarial Learning: A Framework with Convergence Analysis. ICML (2023)
Josh Alman, Zhao Song
Fast Attention Requires Bounded Entries. NeurIPS (2023)
Zhao Song, Mingquan Ye, Junze Yin, Lichen Zhang
Guarantee. ICML (2023)
Zhao Song, Junze Yin, Ruizhe Zhang
Revisiting Quantum Algorithms for Linear Regressions: Quadratic Speedups without Data-Dependent Parameters. CoRR (2023)
Lianke Qin, Zhao Song, Yuanyuan Yang
Efficient SGD Neural Network Training via Sublinear Activated Neuron Identification. CoRR (2023)
Yeqi Gao, Zhao Song, Xin Yang, Ruizhe Zhang
Fast Quantum Algorithm for Attention Computation. CoRR (2023)
Zhenyu Zhang, Ying Sheng, Tianyi Zhou, Tianlong Chen, Lianmin Zheng, Ruisi Cai, Zhao Song, Yuandong Tian, Christopher Ré, Clark W. Barrett, Zhangyang Wang, Beidi Chen
H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models. NeurIPS (2023)
Zhao Song, Mingquan Ye, Lichen Zhang
Streaming Semidefinite Programs: O(√n) Passes, Small Space and Fast Runtime. CoRR (2023)
Josh Alman, Jiehao Liang, Zhao Song, Ruizhe Zhang, Danyang Zhuo
Bypass Exponential Time Preprocessing: Fast Neural Network Training via Weight-Data Correlation Preprocessing. NeurIPS (2023)
Yichuan Deng, Zhihang Li, Sridhar Mahadevan, Zhao Song
Zero-th Order Algorithm for Softmax Attention Optimization. CoRR (2023)
Alexander Munteanu, Simon Omlor, Zhao Song, David P. Woodruff
Bounding the Width of Neural Networks via Coupled Initialization A Worst Case Analysis. ICML (2022)
Lijie Chen, Gillat Kol, Dmitry Paramonov, Raghuvansh Saxena, Zhao Song, Huacheng Yu
Towards Multi-Pass Streaming Lower Bounds for Optimal Approximation of Max-Cut. Electron. Colloquium Comput. Complex. (2022)

Zhao Song.css-eqlu8i{font-size:small;padding-left:4px;}

Publication Activity (10 Years)

Top Topics

Top Venues

Zhao Song