Xiao, Lechao

20 publications

ICML 2025 Scaling Collapse Reveals Universal Dynamics in Compute-Optimally Trained Neural Networks Shikai Qiu, Lechao Xiao, Andrew Gordon Wilson, Jeffrey Pennington, Atish Agarwala

NeurIPS 2024 4+3 Phases of Compute-Optimal Neural Scaling Laws Elliot Paquette, Courtney Paquette, Lechao Xiao, Jeffrey Pennington

TMLR 2024 Beyond Human Data: Scaling Self-Training for Problem-Solving with Language Models Avi Singh, John D Co-Reyes, Rishabh Agarwal, Ankesh Anand, Piyush Patil, Xavier Garcia, Peter J Liu, James Harrison, Jaehoon Lee, Kelvin Xu, Aaron T Parisi, Abhishek Kumar, Alexander A Alemi, Alex Rizkowsky, Azade Nova, Ben Adlam, Bernd Bohnet, Gamaleldin Fathy Elsayed, Hanie Sedghi, Igor Mordatch, Isabelle Simpson, Izzeddin Gur, Jasper Snoek, Jeffrey Pennington, Jiri Hron, Kathleen Kenealy, Kevin Swersky, Kshiteej Mahajan, Laura A Culp, Lechao Xiao, Maxwell Bileschi, Noah Constant, Roman Novak, Rosanne Liu, Tris Warkentin, Yamini Bansal, Ethan Dyer, Behnam Neyshabur, Jascha Sohl-Dickstein, Noah Fiedel

NeurIPSW 2024 Scaling Collapse Reveals Universal Dynamics in Compute-Optimally Trained Neural Networks Shikai Qiu, Atish Agarwala, Jeffrey Pennington, Lechao Xiao

ICML 2024 Scaling Exponents Across Parameterizations and Optimizers Katie E Everett, Lechao Xiao, Mitchell Wortsman, Alexander A Alemi, Roman Novak, Peter J Liu, Izzeddin Gur, Jascha Sohl-Dickstein, Leslie Pack Kaelbling, Jaehoon Lee, Jeffrey Pennington

ICLR 2024 Small-Scale Proxies for Large-Scale Transformer Training Instabilities Mitchell Wortsman, Peter J Liu, Lechao Xiao, Katie E Everett, Alexander A Alemi, Ben Adlam, John D Co-Reyes, Izzeddin Gur, Abhishek Kumar, Roman Novak, Jeffrey Pennington, Jascha Sohl-Dickstein, Kelvin Xu, Jaehoon Lee, Justin Gilmer, Simon Kornblith

COLT 2022 Eigenspace Restructuring: A Principle of Space and Frequency in Neural Networks Lechao Xiao

NeurIPS 2022 Fast Neural Kernel Embeddings for General Activations Insu Han, Amir Zandieh, Jaehoon Lee, Roman Novak, Lechao Xiao, Amin Karbasi

NeurIPS 2022 Precise Learning Curves and Higher-Order Scalings for Dot-Product Kernel Regression Lechao Xiao, Hong Hu, Theodor Misiakiewicz, Yue Lu, Jeffrey Pennington

ICML 2022 Synergy and Symmetry in Deep Learning: Interactions Between the Data, Model, and Inference Algorithm Lechao Xiao, Jeffrey Pennington

NeurIPS 2021 Dataset Distillation with Infinitely Wide Convolutional Networks Timothy Nguyen, Roman Novak, Lechao Xiao, Jaehoon Lee

ICLR 2021 Exploring the Uncertainty Properties of Neural Networks’ Implicit Priors in the Infinite-Width Limit Ben Adlam, Jaehoon Lee, Lechao Xiao, Jeffrey Pennington, Jasper Snoek

ICML 2020 Disentangling Trainability and Generalization in Deep Neural Networks Lechao Xiao, Jeffrey Pennington, Samuel Schoenholz

NeurIPS 2020 Finite Versus Infinite Neural Networks: An Empirical Study Jaehoon Lee, Samuel Schoenholz, Jeffrey Pennington, Ben Adlam, Lechao Xiao, Roman Novak, Jascha Sohl-Dickstein

ICLR 2020 Neural Tangents: Fast and Easy Infinite Neural Networks in Python Roman Novak, Lechao Xiao, Jiri Hron, Jaehoon Lee, Alexander A. Alemi, Jascha Sohl-Dickstein, Samuel S. Schoenholz

ICLR 2020 Provable Benefit of Orthogonal Initialization in Optimizing Deep Linear Networks Wei Hu, Lechao Xiao, Jeffrey Pennington

NeurIPS 2020 The Surprising Simplicity of the Early-Time Learning Dynamics of Neural Networks Wei Hu, Lechao Xiao, Ben Adlam, Jeffrey Pennington

ICLR 2019 Bayesian Deep Convolutional Networks with Many Channels Are Gaussian Processes Roman Novak, Lechao Xiao, Yasaman Bahri, Jaehoon Lee, Greg Yang, Jiri Hron, Daniel A. Abolafia, Jeffrey Pennington, Jascha Sohl-dickstein

NeurIPS 2019 Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descent Jaehoon Lee, Lechao Xiao, Samuel Schoenholz, Yasaman Bahri, Roman Novak, Jascha Sohl-Dickstein, Jeffrey Pennington

ICML 2018 Dynamical Isometry and a Mean Field Theory of CNNs: How to Train 10,000-Layer Vanilla Convolutional Neural Networks Lechao Xiao, Yasaman Bahri, Jascha Sohl-Dickstein, Samuel Schoenholz, Jeffrey Pennington