Defazio, Aaron

23 publications

ICML 2025 PARQ: Piecewise-Affine Regularized Quantization Lisa Jin, Jianhao Ma, Zechun Liu, Andrey Gromov, Aaron Defazio, Lin Xiao

NeurIPS 2025 Purifying Shampoo: Investigating Shampoo's Heuristics by Decomposing Its Preconditioner Runa Eschenhagen, Aaron Defazio, Tsung-Hsien Lee, Richard E. Turner, Hao-Jun Michael Shi

NeurIPS 2024 Directional Smoothness and Gradient Methods: Convergence and Adaptivity Aaron Mishkin, Ahmed Khaled, Yuanhao Wang, Aaron Defazio, Robert M. Gower

ICML 2024 MoMo: Momentum Models for Adaptive Learning Rates Fabian Schaipp, Ruben Ohana, Michael Eickenberg, Aaron Defazio, Robert M. Gower

ICML 2024 Prodigy: An Expeditiously Adaptive Parameter-Free Learner Konstantin Mishchenko, Aaron Defazio

NeurIPS 2024 The Road Less Scheduled Aaron Defazio, Xingyu Yang, Harsh Mehta, Konstantin Mishchenko, Ahmed Khaled, Ashok Cutkosky

NeurIPSW 2023 A Novel Analysis of Gradient Descent Under Directional Smoothness Aaron Mishkin, Ahmed Khaled, Aaron Defazio, Robert M. Gower

ICML 2023 Learning-Rate-Free Learning by D-Adaptation Aaron Defazio, Konstantin Mishchenko

NeurIPS 2023 Mechanic: A Learning Rate Tuner Ashok Cutkosky, Aaron Defazio, Harsh Mehta

JMLR 2022 A Momentumized, Adaptive, Dual Averaged Gradient Method Aaron Defazio, Samy Jelassi

NeurIPSW 2022 Parameter Free Dual Averaging: Optimizing Lipschitz Functions in a Single Pass Aaron Defazio, Konstantin Mishchenko

COLT 2021 Almost Sure Convergence Rates for Stochastic Gradient Descent and Stochastic Heavy Ball Othmane Sebbouh, Robert M Gower, Aaron Defazio

ACML 2021 The Power of Factorial Powers: New Parameter Settings for (Stochastic) Optimization Aaron Defazio, Robert M. Gower

NeurIPS 2020 MRI Banding Removal via Adversarial Training Aaron Defazio, Tullie Murrell, Michael Recht

ICLR 2020 Scaling Laws for the Principled Design, Initialization, and Preconditioning of ReLU Networks Aaron Defazio, Leon Bottou

NeurIPS 2019 On the Curved Geometry of Accelerated Optimization Aaron Defazio

NeurIPS 2019 On the Ineffectiveness of Variance Reduced Optimization for Deep Learning Aaron Defazio, Leon Bottou

NeurIPS 2016 A Simple Practical Accelerated Method for Finite Sums Aaron Defazio

AISTATS 2015 Non-Uniform Stochastic Average Gradient Method for Training Conditional Random Fields Mark Schmidt, Reza Babanezhad, Mohamed Osama Ahmed, Aaron Defazio, Ann Clifton, Anoop Sarkar

ICML 2014 Finito: A Faster, Permutable Incremental Gradient Method for Big Data Problems Aaron Defazio, Justin Domke, Caetano

NeurIPS 2014 SAGA: A Fast Incremental Gradient Method with Support for Non-Strongly Convex Composite Objectives Aaron Defazio, Francis Bach, Simon Lacoste-Julien

NeurIPS 2012 A Convex Formulation for Learning Scale-Free Networks via Submodular Relaxation Aaron Defazio, Tibério S. Caetano

ICML 2012 A Graphical Model Formulation of Collaborative Filtering Neighbourhood Methods with Fast Maximum Entropy Training Aaron Defazio, Tibério S. Caetano