Tengyu Ma

Publication Activity (10 Years)

Years Active: 2011-2024
Publications (10 Years): 184

Top Venues

CoRR

NeurIPS

ICLR

ICML

Publications

Tianle Cai, Xuezhi Wang, Tengyu Ma, Xinyun Chen, Denny Zhou
Large Language Models as Tool Makers. ICLR (2024)
Hong Liu, Zhiyuan Li, David Leo Wright Hall, Percy Liang, Tengyu Ma
Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-training. ICLR (2024)
Zhiyuan Liu, Hong Liu, Denny Zhou, Tengyu Ma
Chain of Thought Empowers Transformers to Solve Inherently Serial Problems. ICLR (2024)
Zhiyuan Li, Hong Liu, Denny Zhou, Tengyu Ma
Chain of Thought Empowers Transformers to Solve Inherently Serial Problems. CoRR (2024)
Neil Band, Xuechen Li, Tengyu Ma, Tatsunori Hashimoto
Linguistic Calibration of Language Models. CoRR (2024)
Arvind V. Mahankali, Tatsunori Hashimoto, Tengyu Ma
One Step of Gradient Descent is Provably the Optimal In-Context Learner with One Layer of Linear Self-Attention. ICLR (2024)
Margalit Glasgow, Colin Wei, Mary Wootters, Tengyu Ma
Max-Margin Works while Large Margin Fails: Generalization without Uniform Convergence. ICLR (2023)
Sang Michael Xie, Shibani Santurkar, Tengyu Ma, Percy Liang
Data Selection for Language Models via Importance Resampling. CoRR (2023)
Kefan Dong, Tengyu Ma
First Steps Toward Understanding the Extrapolation of Nonlinear Models to Unseen Domains. ICLR (2023)
Jeff Z. HaoChen, Tengyu Ma
A theoretical study of inductive biases in contrastive learning. ICLR (2023)
Arvind Mahankali, Haochen Zhang, Kefan Dong, Margalit Glasgow, Tengyu Ma
Beyond NTK with Vanilla Gradient Descent: A Mean-Field Analysis of Neural Networks with Polynomial Width, Samples, and Time. NeurIPS (2023)
Hong Liu, Sang Michael Xie, Zhiyuan Li, Tengyu Ma
Same Pre-training Loss, Better Downstream: Implicit Bias Matters for Language Models. ICML (2023)
Jerry W. Wei, Le Hou, Andrew K. Lampinen, Xiangning Chen, Da Huang, Yi Tay, Xinyun Chen, Yifeng Lu, Denny Zhou, Tengyu Ma, Quoc V. Le
Symbol tuning improves in-context learning in language models. EMNLP (2023)
Khashayar Gatmiry, Zhiyuan Li, Tengyu Ma, Sashank Reddi, Stefanie Jegelka, Ching-Yao Chuang
What is the Inductive Bias of Flatness Regularization? A Study of Deep Matrix Factorization Models. NeurIPS (2023)
Kaiyue Wen, Zhiyuan Li, Tengyu Ma
Sharpness Minimization Algorithms Do Not Only Minimize Sharpness To Achieve Better Generalization. NeurIPS (2023)
Sang Michael Xie, Hieu Pham, Xuanyi Dong, Nan Du, Hanxiao Liu, Yifeng Lu, Percy S. Liang, Quoc V. Le, Tengyu Ma, Adams Wei Yu
DoReMi: Optimizing Data Mixtures Speeds Up Language Model Pretraining. NeurIPS (2023)
Kefan Dong, Tengyu Ma
Asymptotic Instance-Optimal Algorithms for Interactive Decision Making. ICLR (2023)
Sang Michael Xie, Hieu Pham, Xuanyi Dong, Nan Du, Hanxiao Liu, Yifeng Lu, Percy Liang, Quoc V. Le, Tengyu Ma, Adams Wei Yu
DoReMi: Optimizing Data Mixtures Speeds Up Language Model Pretraining. CoRR (2023)
Hong Liu, Zhiyuan Li, David Hall, Percy Liang, Tengyu Ma
Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-training. CoRR (2023)
Jerry W. Wei, Le Hou, Andrew Lampinen, Xiangning Chen, Da Huang, Yi Tay, Xinyun Chen, Yifeng Lu, Denny Zhou, Tengyu Ma, Quoc V. Le
Symbol tuning improves in-context learning in language models. CoRR (2023)
Arvind Mahankali, Tatsunori B. Hashimoto, Tengyu Ma
One Step of Gradient Descent is Provably the Optimal In-Context Learner with One Layer of Linear Self-Attention. CoRR (2023)
Kaiyue Wen, Zhiyuan Li, Tengyu Ma
Sharpness Minimization Algorithms Do Not Only Minimize Sharpness To Achieve Better Generalization. CoRR (2023)
Tianle Cai, Xuezhi Wang, Tengyu Ma, Xinyun Chen, Denny Zhou
Large Language Models as Tool Makers. CoRR (2023)
Sang Michael Xie, Shibani Santurkar, Tengyu Ma, Percy Liang
Data Selection for Language Models via Importance Resampling. NeurIPS (2023)
Kefan Dong, Tengyu Ma
Toward L_∞Recovery of Nonlinear Functions: A Polynomial Sample Complexity Bound for Gaussian Random Fields. COLT (2023)
Khashayar Gatmiry, Zhiyuan Li, Ching-Yao Chuang, Sashank Reddi, Tengyu Ma, Stefanie Jegelka
The Inductive Bias of Flatness Regularization for Deep Matrix Factorization. CoRR (2023)
Ekin Akyürek, Dale Schuurmans, Jacob Andreas, Tengyu Ma, Denny Zhou
What learning algorithm is in-context learning? Investigations with linear models. ICLR (2023)
Jerry W. Wei, Jason Wei, Yi Tay, Dustin Tran, Albert Webson, Yifeng Lu, Xinyun Chen, Hanxiao Liu, Da Huang, Denny Zhou, Tengyu Ma
Larger language models do in-context learning differently. CoRR (2023)
Kefan Dong, Tengyu Ma
-recovery of Nonlinear Functions: A Polynomial Sample Complexity Bound for Gaussian Random Fields. CoRR (2023)
Kaiyue Wen, Tengyu Ma, Zhiyuan Li
How Sharpness-Aware Minimization Minimizes Sharpness? ICLR (2023)
Arvind Mahankali, Jeff Z. HaoChen, Kefan Dong, Margalit Glasgow, Tengyu Ma
Beyond NTK with Vanilla Gradient Descent: A Mean-Field Analysis of Neural Networks with Polynomial Width, Samples, and Time. CoRR (2023)
Ananya Kumar, Aditi Raghunathan, Robbie Jones, Tengyu Ma, Percy Liang
Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distribution. CoRR (2022)
Ling Pan, Longbo Huang, Tengyu Ma, Huazhe Xu
Plan Better Amid Conservatism: Offline Multi-Agent Reinforcement Learning with Actor Rectification. ICML (2022)
Kendrick Shen, Robbie Jones, Ananya Kumar, Sang Michael Xie, Jeff Z. HaoChen, Tengyu Ma, Percy Liang
Connect, Not Collapse: Explaining Contrastive Learning for Unsupervised Domain Adaptation. CoRR (2022)
Jeff Z. HaoChen, Tengyu Ma
A Theoretical Study of Inductive Biases in Contrastive Learning. CoRR (2022)
Hong Liu, Sang Michael Xie, Zhiyuan Li, Tengyu Ma
Same Pre-training Loss, Better Downstream: Implicit Bias Matters for Language Models. CoRR (2022)
Jeff Z. HaoChen, Colin Wei, Ananya Kumar, Tengyu Ma
Beyond Separability: Analyzing the Linear Transferability of Contrastive Representations to Related Subpopulations. CoRR (2022)
Ananya Kumar, Tengyu Ma, Percy Liang, Aditi Raghunathan
Calibrated ensembles can mitigate accuracy tradeoffs under distribution shift. UAI (2022)
Kendrick Shen, Robbie M. Jones, Ananya Kumar, Sang Michael Xie, Jeff Z. HaoChen, Tengyu Ma, Percy Liang
Connect, Not Collapse: Explaining Contrastive Learning for Unsupervised Domain Adaptation. ICML (2022)
Ananya Kumar, Tengyu Ma, Percy Liang, Aditi Raghunathan
Calibrated ensembles can mitigate accuracy tradeoffs under distribution shift. CoRR (2022)
Rong Ge, Tengyu Ma
On the optimization landscape of tensor decompositions. Math. Program. 193 (2) (2022)
Jeff Z. HaoChen, Colin Wei, Ananya Kumar, Tengyu Ma
Beyond Separability: Analyzing the Linear Transferability of Contrastive Representations to Related Subpopulations. NeurIPS (2022)
Yining Chen, Elan Rosenfeld, Mark Sellke, Tengyu Ma, Andrej Risteski
Iterative Feature Matching: Toward Provable Domain Generalization with Logarithmic Environments. NeurIPS (2022)
Kaiyue Wen, Tengyu Ma, Zhiyuan Li
How Does Sharpness-Aware Minimization Minimize Sharpness? CoRR (2022)
Ekin Akyürek, Dale Schuurmans, Jacob Andreas, Tengyu Ma, Denny Zhou
What learning algorithm is in-context learning? Investigations with linear models. CoRR (2022)
Sang Michael Xie, Aditi Raghunathan, Percy Liang, Tengyu Ma
An Explanation of In-context Learning as Implicit Bayesian Inference. ICLR (2022)
Hong Liu, Jeff Z. HaoChen, Adrien Gaidon, Tengyu Ma
Self-supervised Learning is More Robust to Dataset Imbalance. ICLR (2022)
Colin Wei, Yining Chen, Tengyu Ma
Statistically Meaningful Approximation: a Case Study on Approximating Turing Machines with Transformers. NeurIPS (2022)
Haoyuan Cai, Tengyu Ma, Simon S. Du
Near-Optimal Algorithms for Autonomous Exploration and Multi-Goal Stochastic Shortest Path. CoRR (2022)
Kefan Dong, Tengyu Ma
Asymptotic Instance-Optimal Algorithms for Interactive Decision Making. CoRR (2022)
Margalit Glasgow, Colin Wei, Mary Wootters, Tengyu Ma
Max-Margin Works while Large Margin Fails: Generalization without Uniform Convergence. CoRR (2022)
Aviral Kumar, Rishabh Agarwal, Tengyu Ma, Aaron C. Courville, George Tucker, Sergey Levine
DR3: Value-Based Deep Reinforcement Learning Requires Explicit Regularization. ICLR (2022)
Haoyuan Cai, Tengyu Ma, Simon S. Du
Near-Optimal Algorithms for Autonomous Exploration and Multi-Goal Stochastic Shortest Path. ICML (2022)
Kefan Dong, Tengyu Ma
First Steps Toward Understanding the Extrapolation of Nonlinear Models to Unseen Domains. CoRR (2022)
Margalit R. Glasgow, Honglin Yuan, Tengyu Ma
Sharp Bounds for Federated Averaging (Local SGD) and Continuous Perspective. AISTATS (2022)
Garrett Thomas, Yuping Luo, Tengyu Ma
Safe Reinforcement Learning by Imagining the Near Future. CoRR (2022)
Ananya Kumar, Aditi Raghunathan, Robbie Matthew Jones, Tengyu Ma, Percy Liang
Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distribution. ICLR (2022)
Lingxiao Wang, Kevin Huang, Tengyu Ma, Quanquan Gu, Jing Huang
Variance-reduced First-order Meta-learning for Natural Language Processing Tasks. NAACL-HLT (2021)
Kefan Dong, Jiaqi Yang, Tengyu Ma
Provable Model-based Nonlinear Bandit and Reinforcement Learning: Shelve Optimism, Embrace Virtual Curvature. NeurIPS (2021)
Shengjia Zhao, Michael P. Kim, Roshni Sahoo, Tengyu Ma, Stefano Ermon
Calibrating Predictions to Decisions: A Novel Approach to Multi-Class Calibration. CoRR (2021)
Alex Damian, Tengyu Ma, Jason D. Lee
Label Noise SGD Provably Prefers Flat Global Minimizers. CoRR (2021)
Alex Damian, Tengyu Ma, Jason D. Lee
Label Noise SGD Provably Prefers Flat Global Minimizers. NeurIPS (2021)
Kevin Huang, Peng Qi, Guangtao Wang, Tengyu Ma, Jing Huang
Entity and Evidence Guided Document-Level Relation Extraction. RepL4NLP@ACL-IJCNLP (2021)
Colin Wei, Yining Chen, Tengyu Ma
Statistically Meaningful Approximation: a Case Study on Approximating Turing Machines with Transformers. CoRR (2021)
Colin Wei, Kendrick Shen, Yining Chen, Tengyu Ma
Theoretical Analysis of Self-Training with Deep Networks on Unlabeled Data. ICLR (2021)
Haike Xu, Tengyu Ma, Simon S. Du
Fine-Grained Gap-Dependent Bounds for Tabular MDPs via Adaptive Multi-Step Bootstrap. CoRR (2021)
Ling Pan, Longbo Huang, Tengyu Ma, Huazhe Xu
Plan Better Amid Conservatism: Offline Multi-Agent Reinforcement Learning with Actor Rectification. CoRR (2021)
Yining Chen, Elan Rosenfeld, Mark Sellke, Tengyu Ma, Andrej Risteski
Iterative Feature Matching: Toward Provable Domain Generalization with Logarithmic Environments. CoRR (2021)
Garrett Thomas, Yuping Luo, Tengyu Ma
Safe Reinforcement Learning by Imagining the Near Future. NeurIPS (2021)
Kaidi Cao, Yining Chen, Junwei Lu, Nikos Aréchiga, Adrien Gaidon, Tengyu Ma
Heteroskedastic and Imbalanced Deep Learning with Adaptive Regularization. ICLR (2021)
Colin Wei, Sang Michael Xie, Tengyu Ma
Why Do Pretrained Language Models Help in Downstream Tasks? An Analysis of Head and Prompt Tuning. NeurIPS (2021)
Sang Michael Xie, Aditi Raghunathan, Percy Liang, Tengyu Ma
An Explanation of In-context Learning as Implicit Bayesian Inference. CoRR (2021)
Jeff Z. HaoChen, Colin Wei, Adrien Gaidon, Tengyu Ma
Provable Guarantees for Self-Supervised Deep Learning with Spectral Contrastive Loss. NeurIPS (2021)
Kefan Dong, Jiaqi Yang, Tengyu Ma
Provable Model-based Nonlinear Bandit and Reinforcement Learning: Shelve Optimism, Embrace Virtual Curvature. CoRR (2021)
Colin Wei, Sang Michael Xie, Tengyu Ma
Why Do Pretrained Language Models Help in Downstream Tasks? An Analysis of Head and Prompt Tuning. CoRR (2021)
Aviral Kumar, Rishabh Agarwal, Tengyu Ma, Aaron C. Courville, George Tucker, Sergey Levine
DR3: Value-Based Deep Reinforcement Learning Requires Explicit Regularization. CoRR (2021)
Wenxuan Zhou, Kevin Huang, Tengyu Ma, Jing Huang
Document-Level Relation Extraction with Adaptive Thresholding and Localized Context Pooling. AAAI (2021)
Yuping Luo, Tengyu Ma
Learning Barrier Certificates: Towards Safe Reinforcement Learning with Zero Training-time Violations. CoRR (2021)
Shengjia Zhao, Michael P. Kim, Roshni Sahoo, Tengyu Ma, Stefano Ermon
Calibrating Predictions to Decisions: A Novel Approach to Multi-Class Calibration. NeurIPS (2021)
Yuping Luo, Tengyu Ma
Learning Barrier Certificates: Towards Safe Reinforcement Learning with Zero Training-time Violations. NeurIPS (2021)
Zichuan Lin, Jing Huang, Bowen Zhou, Xiaodong He, Tengyu Ma
Joint System-Wise Optimization for Pipeline Goal-Oriented Dialog System. CoRR (2021)
Sang Michael Xie, Tengyu Ma, Percy Liang
Composed Fine-Tuning: Freezing Pre-Trained Denoising Autoencoders for Improved Generalization. ICML (2021)
Margalit Glasgow, Honglin Yuan, Tengyu Ma
Sharp Bounds for Federated Averaging (Local SGD) and Continuous Perspective. CoRR (2021)
Haike Xu, Tengyu Ma, Simon S. Du
Fine-Grained Gap-Dependent Bounds for Tabular MDPs via Adaptive Multi-Step Bootstrap. COLT (2021)
Jeff Z. HaoChen, Colin Wei, Adrien Gaidon, Tengyu Ma
Provable Guarantees for Self-Supervised Deep Learning with Spectral Contrastive Loss. CoRR (2021)
Jeff Z. HaoChen, Colin Wei, Jason D. Lee, Tengyu Ma
Shape Matters: Understanding the Implicit Bias of the Noise Covariance. COLT (2021)
Preetum Nakkiran, Prayaag Venkat, Sham M. Kakade, Tengyu Ma
Optimal Regularization can Mitigate Double Descent. ICLR (2021)
Yining Chen, Haipeng Luo, Tengyu Ma, Chicheng Zhang
Active Online Learning with Hidden Shifting Domains. AISTATS (2021)
Tengyu Ma
Why Do Local Methods Solve Nonconvex Problems? CoRR (2021)
Hong Liu, Jeff Z. HaoChen, Adrien Gaidon, Tengyu Ma
Self-supervised Learning is More Robust to Dataset Imbalance. CoRR (2021)
Sang Michael Xie, Ananya Kumar, Robbie Jones, Fereshte Khani, Tengyu Ma, Percy Liang
In-N-Out: Pre-Training and Self-Training using Auxiliary Information for Out-of-Distribution Robustness. ICLR (2021)
Honglin Yuan, Tengyu Ma
Federated Accelerated Stochastic Gradient Descent. CoRR (2020)
Kefan Dong, Yuping Luo, Tianhe Yu, Chelsea Finn, Tengyu Ma
On the Expressivity of Neural Networks for Deep Reinforcement Learning. ICML (2020)
Yuanzhi Li, Tengyu Ma, Hongyang R. Zhang
Learning Over-Parametrized Two-Layer Neural Networks beyond NTK. COLT (2020)
Xiang Wang, Chenwei Wu, Jason D. Lee, Tengyu Ma, Rong Ge
Beyond Lazy Training for Over-parameterized Tensor Decomposition. NeurIPS (2020)
Yuanzhi Li, Tengyu Ma, Hongyang R. Zhang
Learning Over-Parametrized Two-Layer ReLU Neural Networks beyond NTK. CoRR (2020)
Kevin Huang, Guangtao Wang, Tengyu Ma, Jing Huang
Entity and Evidence Guided Relation Extraction for DocRED. CoRR (2020)
Zichuan Lin, Garrett Thomas, Guangwen Yang, Tengyu Ma
Model-based Adversarial Meta-Reinforcement Learning. CoRR (2020)
Xiang Wang, Chenwei Wu, Jason D. Lee, Tengyu Ma, Rong Ge
Beyond Lazy Training for Over-parameterized Tensor Decomposition. CoRR (2020)
Colin Wei, Sham M. Kakade, Tengyu Ma
The Implicit and Explicit Regularization Effects of Dropout. CoRR (2020)

Tengyu Ma.css-eqlu8i{font-size:small;padding-left:4px;}

Publication Activity (10 Years)

Top Topics

Top Venues

Tengyu Ma