Ma, Tengyu

103 publications

ICML 2025 Non-Asymptotic Length Generalization Thomas Chen, Tengyu Ma, Zhiyuan Li

NeurIPS 2025 Perception Encoder: The Best Visual Embeddings Are Not at the Output of the Network Daniel Bolya, Po-Yao Huang, Peize Sun, Jang Hyun Cho, Andrea Madotto, Chen Wei, Tengyu Ma, Jiale Zhi, Jathushan Rajasegaran, Hanoona Abdul Rasheed, Junke Wang, Marco Monteiro, Hu Xu, Shiyu Dong, Nikhila Ravi, Shang-Wen Li, Piotr Dollar, Christoph Feichtenhofer

NeurIPS 2025 PerceptionLM: Open-Access Data and Models for Detailed Visual Understanding Jang Hyun Cho, Andrea Madotto, Effrosyni Mavroudi, Triantafyllos Afouras, Tushar Nagarajan, Muhammad Maaz, Yale Song, Tengyu Ma, Shuming Hu, Suyog Jain, Miguel Martin, Huiyu Wang, Hanoona Abdul Rasheed, Peize Sun, Po-Yao Huang, Daniel Bolya, Nikhila Ravi, Shashank Jain, Tammy Stark, Seungwhan Moon, Babak Damavandi, Vivian Lee, Andrew Westbury, Salman Khan, Philipp Kraehenbuehl, Piotr Dollar, Lorenzo Torresani, Kristen Grauman, Christoph Feichtenhofer

CVPR 2025 Rethinking Reconstruction and Denoising in the Dark: New Perspective, General Architecture and Beyond Tengyu Ma, Long Ma, Ziye Li, Yuetong Wang, Jinyuan Liu, Chengpei Xu, Risheng Liu

ICLR 2025 SAM 2: Segment Anything in Images and Videos Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman Rädle, Chloe Rolland, Laura Gustafson, Eric Mintun, Junting Pan, Kalyan Vasudev Alwala, Nicolas Carion, Chao-Yuan Wu, Ross Girshick, Piotr Dollar, Christoph Feichtenhofer

ICML 2025 STP: Self-Play LLM Theorem Provers with Iterative Conjecturing and Proving Kefan Dong, Tengyu Ma

ICLR 2025 Understanding Warmup-Stable-Decay Learning Rates: A River Valley Loss Landscape View Kaiyue Wen, Zhiyuan Li, Jason S. Wang, David Leo Wright Hall, Percy Liang, Tengyu Ma

ICLR 2024 Chain of Thought Empowers Transformers to Solve Inherently Serial Problems Zhiyuan Li, Hong Liu, Denny Zhou, Tengyu Ma

NeurIPSW 2024 Formal Theorem Proving by Rewarding LLMs to Decompose Proofs Hierarchically Kefan Dong, Arvind V. Mahankali, Tengyu Ma

ICLR 2024 Large Language Models as Tool Makers Tianle Cai, Xuezhi Wang, Tengyu Ma, Xinyun Chen, Denny Zhou

ICML 2024 Linguistic Calibration of Long-Form Generations Neil Band, Xuechen Li, Tengyu Ma, Tatsunori Hashimoto

ICLR 2024 One Step of Gradient Descent Is Provably the Optimal In-Context Learner with One Layer of Linear Self-Attention Arvind V. Mahankali, Tatsunori Hashimoto, Tengyu Ma

ICLR 2024 Sophia: A Scalable Stochastic Second-Order Optimizer for Language Model Pre-Training Hong Liu, Zhiyuan Li, David Leo Wright Hall, Percy Liang, Tengyu Ma

AAAI 2024 Trash to Treasure: Low-Light Object Detection via Decomposition-and-Aggregation Xiaohan Cui, Long Ma, Tengyu Ma, Jinyuan Liu, Xin Fan, Risheng Liu

ICLR 2023 A Theoretical Study of Inductive Biases in Contrastive Learning Jeff Z. HaoChen, Tengyu Ma

ICLR 2023 Asymptotic Instance-Optimal Algorithms for Interactive Decision Making Kefan Dong, Tengyu Ma

NeurIPS 2023 Beyond NTK with Vanilla Gradient Descent: A Mean-Field Analysis of Neural Networks with Polynomial Width, Samples, and Time Arvind Mahankali, Haochen Zhang, Kefan Dong, Margalit Glasgow, Tengyu Ma

NeurIPS 2023 Data Selection for Language Models via Importance Resampling Sang Michael Xie, Shibani Santurkar, Tengyu Ma, Percy Liang

NeurIPS 2023 DoReMi: Optimizing Data Mixtures Speeds up Language Model Pretraining Sang Michael Xie, Hieu Pham, Xuanyi Dong, Nan Du, Hanxiao Liu, Yifeng Lu, Percy Liang, Quoc V Le, Tengyu Ma, Adams Wei Yu

ICLR 2023 First Steps Toward Understanding the Extrapolation of Nonlinear Models to Unseen Domains Kefan Dong, Tengyu Ma

ICLR 2023 How Sharpness-Aware Minimization Minimizes Sharpness? Kaiyue Wen, Tengyu Ma, Zhiyuan Li

ICLR 2023 Max-Margin Works While Large Margin Fails: Generalization Without Uniform Convergence Margalit Glasgow, Colin Wei, Mary Wootters, Tengyu Ma

ICML 2023 Same Pre-Training Loss, Better Downstream: Implicit Bias Matters for Language Models Hong Liu, Sang Michael Xie, Zhiyuan Li, Tengyu Ma

NeurIPS 2023 Sharpness Minimization Algorithms Do Not Only Minimize Sharpness to Achieve Better Generalization Kaiyue Wen, Zhiyuan Li, Tengyu Ma

ICMLW 2023 Sophia: A Scalable Stochastic Second-Order Optimizer for Language Model Pre-Training Hong Liu, Zhiyuan Li, David Leo Wright Hall, Percy Liang, Tengyu Ma

COLT 2023 Toward L_∞Recovery of Nonlinear Functions: A Polynomial Sample Complexity Bound for Gaussian Random Fields Kefan Dong, Tengyu Ma

NeurIPS 2023 What Is the Inductive Bias of Flatness Regularization? a Study of Deep Matrix Factorization Models Khashayar Gatmiry, Zhiyuan Li, Tengyu Ma, Sashank Reddi, Stefanie Jegelka, Ching-Yao Chuang

ICLR 2023 What Learning Algorithm Is In-Context Learning? Investigations with Linear Models Ekin Akyürek, Dale Schuurmans, Jacob Andreas, Tengyu Ma, Denny Zhou

AISTATS 2022 Sharp Bounds for Federated Averaging (Local SGD) and Continuous Perspective Margalit R. Glasgow, Honglin Yuan, Tengyu Ma

ICLR 2022 An Explanation of In-Context Learning as Implicit Bayesian Inference Sang Michael Xie, Aditi Raghunathan, Percy Liang, Tengyu Ma

NeurIPS 2022 Beyond Separability: Analyzing the Linear Transferability of Contrastive Representations to Related Subpopulations Jeff Z. HaoChen, Colin Wei, Ananya Kumar, Tengyu Ma

UAI 2022 Calibrated Ensembles Can Mitigate Accuracy Tradeoffs Under Distribution Shift Ananya Kumar, Tengyu Ma, Percy Liang, Aditi Raghunathan

ICML 2022 Connect, Not Collapse: Explaining Contrastive Learning for Unsupervised Domain Adaptation Kendrick Shen, Robbie M Jones, Ananya Kumar, Sang Michael Xie, Jeff Z. Haochen, Tengyu Ma, Percy Liang

ICLR 2022 DR3: Value-Based Deep Reinforcement Learning Requires Explicit Regularization Aviral Kumar, Rishabh Agarwal, Tengyu Ma, Aaron Courville, George Tucker, Sergey Levine

ICLR 2022 Fine-Tuning Can Distort Pretrained Features and Underperform Out-of-Distribution Ananya Kumar, Aditi Raghunathan, Robbie Matthew Jones, Tengyu Ma, Percy Liang

NeurIPSW 2022 First Steps Toward Understanding the Extrapolation of Nonlinear Models to Unseen Domains Kefan Dong, Tengyu Ma

NeurIPSW 2022 How Does Sharpness-Aware Minimization Minimizes Sharpness? Kaiyue Wen, Tengyu Ma, Zhiyuan Li

NeurIPS 2022 Iterative Feature Matching: Toward Provable Domain Generalization with Logarithmic Environments Yining Chen, Elan Rosenfeld, Mark Sellke, Tengyu Ma, Andrej Risteski

ICML 2022 Near-Optimal Algorithms for Autonomous Exploration and Multi-Goal Stochastic Shortest Path Haoyuan Cai, Tengyu Ma, Simon Du

ICML 2022 Plan Better amid Conservatism: Offline Multi-Agent Reinforcement Learning with Actor Rectification Ling Pan, Longbo Huang, Tengyu Ma, Huazhe Xu

ICLR 2022 Self-Supervised Learning Is More Robust to Dataset Imbalance Hong Liu, Jeff Z. HaoChen, Adrien Gaidon, Tengyu Ma

NeurIPS 2022 Statistically Meaningful Approximation: A Case Study on Approximating Turing Machines with Transformers Colin Wei, Yining Chen, Tengyu Ma

CVPR 2022 Toward Fast, Flexible, and Robust Low-Light Image Enhancement Long Ma, Tengyu Ma, Risheng Liu, Xin Fan, Zhongxuan Luo

AISTATS 2021 Active Online Learning with Hidden Shifting Domains Yining Chen, Haipeng Luo, Tengyu Ma, Chicheng Zhang

NeurIPSW 2021 Calibrated Ensembles: A Simple Way to Mitigate ID-OOD Accuracy Tradeoffs Ananya Kumar, Aditi Raghunathan, Tengyu Ma, Percy Liang

NeurIPS 2021 Calibrating Predictions to Decisions: A Novel Approach to Multi-Class Calibration Shengjia Zhao, Michael Kim, Roshni Sahoo, Tengyu Ma, Stefano Ermon

ICML 2021 Composed Fine-Tuning: Freezing Pre-Trained Denoising Autoencoders for Improved Generalization Sang Michael Xie, Tengyu Ma, Percy Liang

NeurIPSW 2021 DR3: Value-Based Deep Reinforcement Learning Requires Explicit Regularization Aviral Kumar, Rishabh Agarwal, Tengyu Ma, Aaron Courville, George Tucker, Sergey Levine

AAAI 2021 Document-Level Relation Extraction with Adaptive Thresholding and Localized Context Pooling Wenxuan Zhou, Kevin Huang, Tengyu Ma, Jing Huang

COLT 2021 Fine-Grained Gap-Dependent Bounds for Tabular MDPs via Adaptive Multi-Step Bootstrap Haike Xu, Tengyu Ma, Simon Du

ICLR 2021 Heteroskedastic and Imbalanced Deep Learning with Adaptive Regularization Kaidi Cao, Yining Chen, Junwei Lu, Nikos Arechiga, Adrien Gaidon, Tengyu Ma

ICLR 2021 In-N-Out: Pre-Training and Self-Training Using Auxiliary Information for Out-of-Distribution Robustness Sang Michael Xie, Ananya Kumar, Robbie Jones, Fereshte Khani, Tengyu Ma, Percy Liang

NeurIPS 2021 Label Noise SGD Provably Prefers Flat Global Minimizers Alex Damian, Tengyu Ma, Jason Lee

NeurIPS 2021 Learning Barrier Certificates: Towards Safe Reinforcement Learning with Zero Training-Time Violations Yuping Luo, Tengyu Ma

ICLR 2021 Optimal Regularization Can Mitigate Double Descent Preetum Nakkiran, Prayaag Venkat, Sham M. Kakade, Tengyu Ma

NeurIPSW 2021 Plan Better amid Conservatism: Offline Multi-Agent Reinforcement Learning with Actor Rectification Ling Pan, Longbo Huang, Tengyu Ma, Huazhe Xu

NeurIPS 2021 Provable Guarantees for Self-Supervised Deep Learning with Spectral Contrastive Loss Jeff Z. HaoChen, Colin Wei, Adrien Gaidon, Tengyu Ma

NeurIPS 2021 Provable Model-Based Nonlinear Bandit and Reinforcement Learning: Shelve Optimism, Embrace Virtual Curvature Kefan Dong, Jiaqi Yang, Tengyu Ma

NeurIPS 2021 Safe Reinforcement Learning by Imagining the near Future Garrett Thomas, Yuping Luo, Tengyu Ma

NeurIPSW 2021 Self-Supervised Learning Is More Robust to Dataset Imbalance Hong Liu, Jeff Z. HaoChen, Adrien Gaidon, Tengyu Ma

COLT 2021 Shape Matters: Understanding the Implicit Bias of the Noise Covariance Jeff Z. HaoChen, Colin Wei, Jason Lee, Tengyu Ma

ICLR 2021 Theoretical Analysis of Self-Training with Deep Networks on Unlabeled Data Colin Wei, Kendrick Shen, Yining Chen, Tengyu Ma

NeurIPS 2021 Why Do Pretrained Language Models Help in Downstream Tasks? an Analysis of Head and Prompt Tuning Colin Wei, Sang Michael Xie, Tengyu Ma

ICMLW 2020 Active Online Domain Adaptation Yining Chen, Haipeng Luo, Tengyu Ma, Chicheng Zhang

NeurIPS 2020 Beyond Lazy Training for Over-Parameterized Tensor Decomposition Xiang Wang, Chenwei Wu, Jason Lee, Tengyu Ma, Rong Ge

NeurIPS 2020 Federated Accelerated Stochastic Gradient Descent Honglin Yuan, Tengyu Ma

ICLR 2020 Improved Sample Complexities for Deep Neural Networks and Robust Classification via an All-Layer Margin Colin Wei, Tengyu Ma

ICML 2020 Individual Calibration with Randomized Forecasting Shengjia Zhao, Tengyu Ma, Stefano Ermon

COLT 2020 Learning Over-Parametrized Two-Layer Neural Networks Beyond NTK Yuanzhi Li, Tengyu Ma, Hongyang R. Zhang

ICLR 2020 Learning Self-Correctable Policies and Value Functions from Demonstrations with Negative Sampling Yuping Luo, Huazhe Xu, Tengyu Ma

NeurIPS 2020 MOPO: Model-Based Offline Policy Optimization Tianhe Yu, Garrett Thomas, Lantao Yu, Stefano Ermon, James Y Zou, Sergey Levine, Chelsea Finn, Tengyu Ma

NeurIPS 2020 Model-Based Adversarial Meta-Reinforcement Learning Zichuan Lin, Garrett Thomas, Guangwen Yang, Tengyu Ma

ICML 2020 On the Expressivity of Neural Networks for Deep Reinforcement Learning Kefan Dong, Yuping Luo, Tianhe Yu, Chelsea Finn, Tengyu Ma

ECCV 2020 Robust and On-the-Fly Dataset Denoising for Image Classification Jiaming Song, Yann Dauphin, Michael Auli, Tengyu Ma

NeurIPS 2020 Self-Training Avoids Using Spurious Features Under Domain Shift Yining Chen, Colin Wei, Ananya Kumar, Tengyu Ma

ICML 2020 The Implicit and Explicit Regularization Effects of Dropout Colin Wei, Sham Kakade, Tengyu Ma

ICML 2020 Understanding Self-Training for Gradual Domain Adaptation Ananya Kumar, Tengyu Ma, Percy Liang

ICLR 2019 Algorithmic Framework for Model-Based Deep Reinforcement Learning with Theoretical Guarantees Yuping Luo, Huazhe Xu, Yuanzhi Li, Yuandong Tian, Trevor Darrell, Tengyu Ma

ICLR 2019 Approximability of Discriminators Implies Diversity in GANs Yu Bai, Tengyu Ma, Andrej Risteski

NeurIPS 2019 Data-Dependent Sample Complexity of Deep Neural Networks via Lipschitz Augmentation Colin Wei, Tengyu Ma

ICLR 2019 Fixup Initialization: Residual Learning Without Normalization Hongyi Zhang, Yann N. Dauphin, Tengyu Ma

NeurIPS 2019 Learning Imbalanced Datasets with Label-Distribution-Aware Margin Loss Kaidi Cao, Colin Wei, Adrien Gaidon, Nikos Arechiga, Tengyu Ma

COLT 2019 On the Performance of Thompson Sampling on Logistic Bandits Shi Dong, Tengyu Ma, Benjamin Van Roy

NeurIPS 2019 Regularization Matters: Generalization and Optimization of Neural Nets V.s. Their Induced Kernel Colin Wei, Jason Lee, Qiang Liu, Tengyu Ma

NeurIPS 2019 Towards Explaining the Regularization Effect of Initial Large Learning Rate in Training Neural Networks Yuanzhi Li, Colin Wei, Tengyu Ma

NeurIPS 2019 Verified Uncertainty Calibration Ananya Kumar, Percy Liang, Tengyu Ma

COLT 2018 Algorithmic Regularization in Over-Parameterized Matrix Sensing and Neural Networks with Quadratic Activations Yuanzhi Li, Tengyu Ma, Hongyang Zhang

JMLR 2018 Gradient Descent Learns Linear Dynamical Systems Moritz Hardt, Tengyu Ma, Benjamin Recht

ICLR 2018 Learning One-Hidden-Layer Neural Networks with Landscape Design Rong Ge, Jason D. Lee, Tengyu Ma

ICLR 2017 A Simple but Tough-to-Beat Baseline for Sentence Embeddings Sanjeev Arora, Yingyu Liang, Tengyu Ma

JMLR 2017 Distributed Stochastic Variance Reduced Gradient Methods by Sampling Extra Data with Replacement Jason D. Lee, Qihang Lin, Tengyu Ma, Tianbao Yang

ICML 2017 Generalization and Equilibrium in Generative Adversarial Nets (GANs) Sanjeev Arora, Rong Ge, Yingyu Liang, Tengyu Ma, Yi Zhang

ICLR 2017 Identity Matters in Deep Learning Moritz Hardt, Tengyu Ma

COLT 2017 On the Ability of Neural Nets to Express Distributions Holden Lee, Rong Ge, Tengyu Ma, Andrej Risteski, Sanjeev Arora

NeurIPS 2017 On the Optimization Landscape of Tensor Decompositions Rong Ge, Tengyu Ma

NeurIPS 2016 A Non-Generative Framework and Convex Relaxations for Unsupervised Learning Elad Hazan, Tengyu Ma

NeurIPS 2016 Matrix Completion Has No Spurious Local Minimum Rong Ge, Jason Lee, Tengyu Ma

ICML 2016 Provable Algorithms for Inference in Topic Models Sanjeev Arora, Rong Ge, Frederic Koehler, Tengyu Ma, Ankur Moitra

ICML 2015 Online Learning of Eigenvectors Dan Garber, Elad Hazan, Tengyu Ma

COLT 2015 Simple, Efficient, and Neural Algorithms for Sparse Coding Sanjeev Arora, Rong Ge, Tengyu Ma, Ankur Moitra

NeurIPS 2015 Sum-of-Squares Lower Bounds for Sparse PCA Tengyu Ma, Avi Wigderson

NeurIPS 2014 On Communication Cost of Distributed Statistical Estimation and Dimensionality Ankit Garg, Tengyu Ma, Huy Nguyen

ICML 2014 Provable Bounds for Learning Some Deep Representations Sanjeev Arora, Aditya Bhaskara, Rong Ge, Tengyu Ma