Li, Zhiyuan

68 publications

ICLR 2025 A Coefficient Makes SVRG Effective Yida Yin, Zhiqiu Xu, Zhiyuan Li, Trevor Darrell, Zhuang Liu

COLT 2025 A Theory of Learning with Autoregressive Chain of Thought Nirmit Joshi, Gal Vardi, Adam Block, Surbhi Goel, Zhiyuan Li, Theodor Misiakiewicz, Nathan Srebro

ICLR 2025 Adam Exploits $\ell_\infty$-Geometry of Loss Landscape via Coordinate-Wise Adaptivity Shuo Xie, Mohamad Amin Mohamadi, Zhiyuan Li

AAAI 2025 AgentMixer: Multi-Agent Correlated Policy Factorization Zhiyuan Li, Wenshuai Zhao, Lijun Wu, Joni Pajarinen

ICLR 2025 Chain-of-Thought Provably Enables Learning the (Otherwise) Unlearnable Chenxiao Yang, Zhiyuan Li, David Wipf

ICCV 2025 Find a Scapegoat: Poisoning Membership Inference Attack and Defense to Federated Learning Wenjin Mo, Zhiyuan Li, Minghong Fang, Mingwei Fang

ICML 2025 Learning Progress Driven Multi-Agent Curriculum Wenshuai Zhao, Zhiyuan Li, Joni Pajarinen

ICML 2025 Non-Asymptotic Length Generalization Thomas Chen, Tengyu Ma, Zhiyuan Li

NeurIPS 2025 On Learning Verifiers and Implications to Chain-of-Thought Reasoning Maria Florina Balcan, Avrim Blum, Zhiyuan Li, Dravyansh Sharma

ICML 2025 PENCIL: Long Thoughts with Short Memory Chenxiao Yang, Nathan Srebro, David Mcallester, Zhiyuan Li

ICLRW 2025 PENCIL: Long Thoughts with Short Memory Chenxiao Yang, Nathan Srebro, David McAllester, Zhiyuan Li

ICLR 2025 Reasoning with Latent Thoughts: On the Power of Looped Transformers Nikunj Saunshi, Nishanth Dikkala, Zhiyuan Li, Sanjiv Kumar, Sashank J. Reddi

ICML 2025 Structured Preconditioners in Adaptive Optimization: A Unified Analysis Shuo Xie, Tianhao Wang, Sashank J. Reddi, Sanjiv Kumar, Zhiyuan Li

ICLR 2025 Understanding Warmup-Stable-Decay Learning Rates: A River Valley Loss Landscape View Kaiyue Wen, Zhiyuan Li, Jason S. Wang, David Leo Wright Hall, Percy Liang, Tengyu Ma

ICML 2025 Weak-to-Strong Generalization Even in Random Feature Networks, Provably Marko Medvedev, Kaifeng Lyu, Dingli Yu, Sanjeev Arora, Zhiyuan Li, Nathan Srebro

ICMLW 2024 Adam Exploits $\ell_\infty$-Geometry of Loss Landscape via Coordinate-Wise Adaptivity Shuo Xie, Mohamad Amin Mohamadi, Zhiyuan Li

ICMLW 2024 An In-Context Learning Theoretic Analysis of Chain-of-Thought Chenxiao Yang, Zhiyuan Li, David Wipf

AAAI 2024 Backpropagation Through Agents Zhiyuan Li, Wenshuai Zhao, Lijun Wu, Joni Pajarinen

ICLR 2024 Chain of Thought Empowers Transformers to Solve Inherently Serial Problems Zhiyuan Li, Hong Liu, Denny Zhou, Tengyu Ma

WACV 2024 Complex Organ Mask Guided Radiology Report Generation Tiancheng Gu, Dongnan Liu, Zhiyuan Li, Weidong Cai

ICLR 2024 Dichotomy of Early and Late Phase Implicit Biases Can Provably Induce Grokking Kaifeng Lyu, Jikai Jin, Zhiyuan Li, Simon Shaolei Du, Jason D. Lee, Wei Hu

ICLR 2024 Fast Equilibrium of SGD in Generic Situations Zhiyuan Li, Yi Wang, Zhiren Wang

ICML 2024 Implicit Bias of AdamW: $\ell_∞$-Norm Constrained Optimization Shuo Xie, Zhiyuan Li

ICML 2024 Optimistic Multi-Agent Policy Gradient Wenshuai Zhao, Yi Zhao, Zhiyuan Li, Juho Kannala, Joni Pajarinen

CVPRW 2024 Real-Time 4k Super-Resolution of Compressed AVIF Images. AIS 2024 Challenge Survey Marcos V. Conde, Zhijun Lei, Wen Li, Ioannis Katsavounidis, Radu Timofte, Min Yan, Xin Liu, Qian Wang, Xiaoqian Ye, Zhan Du, Tiansen Zhang, Zhiyuan Li, Hao Wei, Chenyang Ge, Jiangtao Lv, Long Sun, Jinshan Pan, Jiangxin Dong, Jinhui Tang, Menghan Zhou, Yiqiang Yan, Kihwan Yoon, Ganzorig Gankhuyag, Jae-Hyeon Lee, Ui-Jin Choi, Hyeon-Cheol Moon, Tae Hyun Jeong, Yoonmo Yang, Jae-Gon Kim, Jinwoo Jeong, Sunjei Kim, Xintao Qiu, Yuanbo Zhou, Kongxian Wu, Xinwei Dai, Hui Tang, Wei Deng, Qingquan Gao, Tong Tong, Long Peng, Jiaming Guo, Xin Di, Bohao Liao, Zhibo Du, Peize Xia, Renjing Pei, Yang Wang, Yang Cao, Zhengjun Zha, Bingnan Han, Hongyuan Yu, Zhuoyuan Wu, Cheng Wan, Yuqing Liu, Haodong Yu, Jizhe Li, Zhijuan Huang, Yuan Huang, Yajun Zou, Xianyu Guan, Qi Jia, Heng Zhang, Xuanwu Yin, Kunlong Zuo, Dongyang Zhang, Tianle Liu, Huaian Chen, Yi Jin

ICML 2024 Simplicity Bias via Global Convergence of Sharpness Minimization Khashayar Gatmiry, Zhiyuan Li, Sashank J. Reddi, Stefanie Jegelka

ICLR 2024 Sophia: A Scalable Stochastic Second-Order Optimizer for Language Model Pre-Training Hong Liu, Zhiyuan Li, David Leo Wright Hall, Percy Liang, Tengyu Ma

ICLR 2024 The Marginal Value of Momentum for Small Learning Rate SGD Runzhe Wang, Sadhika Malladi, Tianhao Wang, Kaifeng Lyu, Zhiyuan Li

ICML 2024 Why Do You Grok? a Theoretical Analysis on Grokking Modular Addition Mohamad Amin Mohamadi, Zhiyuan Li, Lei Wu, Danica J. Sutherland

ICLR 2023 Continual Unsupervised Disentangling of Self-Organizing Representations Zhiyuan Li, Xiajun Jiang, Ryan Missel, Prashnna Kumar Gyawali, Nilesh Kumar, Linwei Wang

NeurIPSW 2023 Grokking Modular Arithmetic Can Be Explained by Margin Maximization Mohamad Amin Mohamadi, Zhiyuan Li, Lei Wu, Danica Sutherland

ICLR 2023 How Sharpness-Aware Minimization Minimizes Sharpness? Kaiyue Wen, Tengyu Ma, Zhiyuan Li

CVPRW 2023 NTIRE 2023 Challenge on Image Super-Resolution (×4): Methods and Results Yulun Zhang, Kai Zhang, Zheng Chen, Yawei Li, Radu Timofte, Junpei Zhang, Kexin Zhang, Rui Peng, Yanbiao Ma, Licheng Jia, Huaibo Huang, Xiaoqiang Zhou, Yuang Ai, Ran He, Yajun Qiu, Qiang Zhu, Pengfei Li, Qianhui Li, Shuyuan Zhu, Dafeng Zhang, Jia Li, Fan Wang, Chunmiao Li, TaeHyung Kim, Jungkeong Kil, Eon Kim, Yeonseung Yu, Beomyeol Lee, Subin Lee, Seokjae Lim, Somi Chae, Heungjun Choi, Zhi-Kai Huang, YiChung Chen, Yuan-Chun Chiang, Hao-Hsiang Yang, Wei-Ting Chen, Hua-En Chang, I-Hsiang Chen, Chia-Hsuan Hsieh, Sy-Yen Kuo, Ui-Jin Choi, Marcos V. Conde, Sunder Ali Khowaja, Jiseok Yoon, Ik Hyun Lee, Garas Gendy, Nabil Sabor, Jingchao Hou, Guanghui He, Zhao Zhang, Baiang Li, Huan Zheng, Suiyi Zhao, Yangcheng Gao, Yanyan Wei, Jiahuan Ren, Jiayu Wei, Yanfeng Li, Jia Sun, Zhanyi Cheng, Zhiyuan Li, Xu Yao, Xinyi Wang, Danxu Li, Xuan Cui, Jun Cao, Cheng Li, Jianbin Zheng, Anjali Sarvaiya, Kalpesh Prajapati, Ratnadeep Patra, Pragnesh Barik, Chaitanya Rathod, Kishor P. Upla, Kiran B. Raja, Raghavendra Ramachandra, Christoph Busch

ICML 2023 Same Pre-Training Loss, Better Downstream: Implicit Bias Matters for Language Models Hong Liu, Sang Michael Xie, Zhiyuan Li, Tengyu Ma

ICLR 2023 Sequential Latent Variable Models for Few-Shot High-Dimensional Time-Series Forecasting Xiajun Jiang, Ryan Missel, Zhiyuan Li, Linwei Wang

NeurIPS 2023 Sharpness Minimization Algorithms Do Not Only Minimize Sharpness to Achieve Better Generalization Kaiyue Wen, Zhiyuan Li, Tengyu Ma

ICMLW 2023 Sophia: A Scalable Stochastic Second-Order Optimizer for Language Model Pre-Training Hong Liu, Zhiyuan Li, David Leo Wright Hall, Percy Liang, Tengyu Ma

ICML 2023 Understanding Incremental Learning of Gradient Descent: A Fine-Grained Analysis of Matrix Sensing Jikai Jin, Zhiyuan Li, Kaifeng Lyu, Simon Shaolei Du, Jason D. Lee

NeurIPS 2023 What Is the Inductive Bias of Flatness Regularization? a Study of Deep Matrix Factorization Models Khashayar Gatmiry, Zhiyuan Li, Tengyu Ma, Sashank Reddi, Stefanie Jegelka, Ching-Yao Chuang

NeurIPS 2022 Fast Mixing of Stochastic Gradient Descent with Normalization and Weight Decay Zhiyuan Li, Tianhao Wang, Dingli Yu

NeurIPSW 2022 How Does Sharpness-Aware Minimization Minimizes Sharpness? Kaiyue Wen, Tengyu Ma, Zhiyuan Li

NeurIPS 2022 Implicit Bias of Gradient Descent on Reparametrized Models: On Equivalence to Mirror Descent Zhiyuan Li, Tianhao Wang, Jason Lee, Sanjeev Arora

ICML 2022 Robust Training of Neural Networks Using Scale Invariant Architectures Zhiyuan Li, Srinadh Bhojanapalli, Manzil Zaheer, Sashank Reddi, Sanjiv Kumar

ICML 2022 Understanding Gradient Descent on the Edge of Stability in Deep Learning Sanjeev Arora, Zhiyuan Li, Abhishek Panigrahi

NeurIPS 2022 Understanding the Generalization Benefit of Normalization Layers: Sharpness Reduction Kaifeng Lyu, Zhiyuan Li, Sanjeev Arora

ICLR 2022 What Happens After SGD Reaches Zero Loss? --a Mathematical Framework Zhiyuan Li, Tianhao Wang, Sanjeev Arora

ICCV 2021 DeFRCN: Decoupled Faster R-CNN for Few-Shot Object Detection Limeng Qiao, Yuxuan Zhao, Zhiyuan Li, Xi Qiu, Jianan Wu, Chi Zhang

NeurIPS 2021 Gradient Descent on Two-Layer Nets: Margin Maximization and Simplicity Bias Kaifeng Lyu, Zhiyuan Li, Runzhe Wang, Sanjeev Arora

NeurIPS 2021 On the Validity of Modeling SGD with Stochastic Differential Equations (SDEs) Zhiyuan Li, Sadhika Malladi, Sanjeev Arora

WACV 2021 Regional Attention Networks with Context-Aware Fusion for Group Emotion Recognition Ahmed Shehab Khan, Zhiyuan Li, Jie Cai, Yan Tong

ICML 2021 Risk Bounds and Rademacher Complexity in Batch Reinforcement Learning Yaqi Duan, Chi Jin, Zhiyuan Li

ICLR 2021 Towards Resolving the Implicit Bias of Gradient Descent for Matrix Factorization: Greedy Low-Rank Learning Zhiyuan Li, Yuping Luo, Kaifeng Lyu

UAI 2021 When Is Particle Filtering Efficient for Planning in Partially Observed Linear Dynamical Systems? Simon S. Du, Wei Hu, Zhiyuan Li, Ruoqi Shen, Zhao Song, Jiajun Wu

ICLR 2021 Why Are Convolutional Nets More Sample-Efficient than Fully-Connected Nets? Zhiyuan Li, Yi Zhang, Sanjeev Arora

ICLR 2020 An Exponential Learning Rate Schedule for Deep Learning Zhiyuan Li, Sanjeev Arora

ICLR 2020 Harnessing the Power of Infinitely Wide Deep Nets on Small-Data Tasks Sanjeev Arora, Simon S. Du, Zhiyuan Li, Ruslan Salakhutdinov, Ruosong Wang, Dingli Yu

NeurIPS 2020 Implicit Regularization and Convergence for Weight Normalization Xiaoxia Wu, Edgar Dobriban, Tongzheng Ren, Shanshan Wu, Zhiyuan Li, Suriya Gunasekar, Rachel Ward, Qiang Liu

ICLR 2020 Progressive Learning and Disentanglement of Hierarchical Representations Zhiyuan Li, Jaideep Vitthal Murkute, Prashnna Kumar Gyawali, Linwei Wang

NeurIPS 2020 Reconciling Modern Deep Learning with Traditional Optimization Analyses: The Intrinsic Learning Rate Zhiyuan Li, Kaifeng Lyu, Sanjeev Arora

ICLR 2020 Simple and Effective Regularization Methods for Training on Noisily Labeled Data with Generalization Guarantee Wei Hu, Zhiyuan Li, Dingli Yu

NeurIPS 2019 Explaining Landscape Connectivity of Low-Cost Solutions for Multilayer Nets Rohith Kuditipudi, Xiang Wang, Holden Lee, Yi Zhang, Zhiyuan Li, Wei Hu, Rong Ge, Sanjeev Arora

ICML 2019 Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networks Sanjeev Arora, Simon Du, Wei Hu, Zhiyuan Li, Ruosong Wang

NeurIPS 2019 On Exact Computation with an Infinitely Wide Neural Net Sanjeev Arora, Simon S Du, Wei Hu, Zhiyuan Li, Ruslan Salakhutdinov, Ruosong Wang

ICLR 2019 The Role of Over-Parametrization in Generalization of Neural Networks Behnam Neyshabur, Zhiyuan Li, Srinadh Bhojanapalli, Yann LeCun, Nathan Srebro

ICLR 2019 Theoretical Analysis of Auto Rate-Tuning by Batch Normalization Sanjeev Arora, Zhiyuan Li, Kaifeng Lyu

NeurIPS 2018 Online Improper Learning with an Approximation Oracle Elad Hazan, Wei Hu, Yuanzhi Li, Zhiyuan Li

NeurIPS 2016 Learning in Games: Robustness of Fast Convergence Dylan J Foster, Zhiyuan Li, Thodoris Lykouris, Karthik Sridharan, Eva Tardos

NeurIPS 2016 Solving Marginal MAP Problems with NP Oracles and Parity Constraints Yexiang Xue, Zhiyuan Li, Stefano Ermon, Carla P. Gomes, Bart Selman