Aaron Defazio

Publication Activity (10 Years)

Years Active: 2012-2024
Publications (10 Years): 38

Top Topics

Delta Bar Delta

Covariate Shift

Gradient Method

Top Venues

Publications

Aaron Mishkin, Ahmed Khaled, Yuanhao Wang, Aaron Defazio, Robert M. Gower
Directional Smoothness and Gradient Methods: Convergence and Adaptivity. CoRR (2024)
Aaron Defazio, Xingyu Yang, Harsh Mehta, Konstantin Mishchenko, Ahmed Khaled, Ashok Cutkosky
The Road Less Scheduled. CoRR (2024)
Fabian Schaipp, Ruben Ohana, Michael Eickenberg, Aaron Defazio, Robert M. Gower
MoMo: Momentum Models for Adaptive Learning Rates. CoRR (2023)
Aaron Defazio, Konstantin Mishchenko
Learning-Rate-Free Learning by D-Adaptation. CoRR (2023)
Ashok Cutkosky, Aaron Defazio, Harsh Mehta
Mechanic: A Learning Rate Tuner. CoRR (2023)
Ashok Cutkosky, Aaron Defazio, Harsh Mehta
Mechanic: A Learning Rate Tuner. NeurIPS (2023)
Konstantin Mishchenko, Aaron Defazio
Prodigy: An Expeditiously Adaptive Parameter-Free Learner. CoRR (2023)
Aaron Defazio, Ashok Cutkosky, Harsh Mehta, Konstantin Mishchenko
When, Why and How Much? Adaptive Learning Rate Scheduling by Refinement. CoRR (2023)
Aaron Defazio, Konstantin Mishchenko
Learning-Rate-Free Learning by D-Adaptation. ICML (2023)
Aaron Defazio, Samy Jelassi
A Momentumized, Adaptive, Dual Averaged Gradient Method. J. Mach. Learn. Res. 23 (2022)
Aaron Defazio, Léon Bottou
A scaling calculus for the design and initialization of ReLU networks. Neural Comput. Appl. 34 (17) (2022)
Aaron Defazio, Baoyu Zhou, Lin Xiao
Grad-GradaGrad? A Non-Monotone Adaptive Stochastic Gradient Method. CoRR (2022)
Aaron Defazio, Mark Tygert, Rachel Ward, Jure Zbontar
Compressed sensing with a jackknife and a bootstrap. J. Data Sci. Stat. Vis. 2 (4) (2022)
Robert M. Gower, Aaron Defazio, Michael G. Rabbat
Stochastic Polyak Stepsize with a Moving Target. CoRR (2021)
Aaron Defazio, Robert M. Gower
The Power of Factorial Powers: New Parameter settings for (Stochastic) Optimization. ACML (2021)
Aaron Defazio, Samy Jelassi
Adaptivity without Compromise: A Momentumized, Adaptive, Dual Averaged Gradient Method for Stochastic Optimization. CoRR (2021)
Othmane Sebbouh, Robert M. Gower, Aaron Defazio
Almost sure convergence rates for Stochastic Gradient Descent and Stochastic Heavy Ball. COLT (2021)
Othmane Sebbouh, Robert M. Gower, Aaron Defazio
On the convergence of the Stochastic Heavy Ball Method. CoRR (2020)
Aaron Defazio, Tullie Murrell, Michael P. Recht
MRI Banding Removal via Adversarial Training. CoRR (2020)
Florian Knoll, Tullie Murrell, Anuroop Sriram, Nafissa Yakubova, Jure Zbontar, Michael G. Rabbat, Aaron Defazio, Matthew J. Muckley, Daniel K. Sodickson, C. Lawrence Zitnick, Michael P. Recht
Advancing machine learning for MR image reconstruction with an open competition: Overview of the 2019 fastMRI challenge. CoRR (2020)
Anuroop Sriram, Jure Zbontar, Tullie Murrell, Aaron Defazio, C. Lawrence Zitnick, Nafissa Yakubova, Florian Knoll, Patricia M. Johnson
End-to-End Variational Networks for Accelerated MRI Reconstruction. MICCAI (2) (2020)
Aaron Defazio, Tullie Murrell, Michael P. Recht
MRI Banding Removal via Adversarial Training. NeurIPS (2020)
Anuroop Sriram, Jure Zbontar, Tullie Murrell, C. Lawrence Zitnick, Aaron Defazio, Daniel K. Sodickson
GrappaNet: Combining Parallel Imaging With Deep Learning for Multi-Coil MRI Reconstruction. CVPR (2020)
Aaron Defazio
Understanding the Role of Momentum in Non-Convex Optimization: Practical Insights from a Lyapunov Analysis. CoRR (2020)
Anuroop Sriram, Jure Zbontar, Tullie Murrell, Aaron Defazio, C. Lawrence Zitnick, Nafissa Yakubova, Florian Knoll, Patricia M. Johnson
End-to-End Variational Networks for Accelerated MRI Reconstruction. CoRR (2020)
Aaron Defazio, Robert M. Gower
Factorial Powers for Stochastic Optimization. CoRR (2020)
Samy Jelassi, Aaron Defazio
Dual Averaging is Surprisingly Effective for Deep Learning Optimization. CoRR (2020)
Aaron Defazio
Offset Masking Improves Deep Learning based Accelerated MRI Reconstructions. CoRR (2019)
Aaron Defazio, Léon Bottou
On the Ineffectiveness of Variance Reduced Optimization for Deep Learning. NeurIPS (2019)
Aaron Defazio, Léon Bottou
Scaling Laws for the Principled Design, Initialization and Preconditioning of ReLU Networks. CoRR (2019)
Aaron Defazio
On the Curved Geometry of Accelerated Optimization. NeurIPS (2019)
Anuroop Sriram, Jure Zbontar, Tullie Murrell, C. Lawrence Zitnick, Aaron Defazio, Daniel K. Sodickson
GrappaNet: Combining Parallel Imaging with Deep Learning for Multi-Coil MRI Reconstruction. CoRR (2019)
Jure Zbontar, Florian Knoll, Anuroop Sriram, Matthew J. Muckley, Mary Bruno, Aaron Defazio, Marc Parente, Krzysztof J. Geras, Joe Katsnelson, Hersh Chandarana, Zizhao Zhang, Michal Drozdzal, Adriana Romero, Michael G. Rabbat, Pascal Vincent, James Pinkerton, Duo Wang, Nafissa Yakubova, Erich Owens, C. Lawrence Zitnick, Michael P. Recht, Daniel K. Sodickson, Yvonne W. Lui
fastMRI: An Open Dataset and Benchmarks for Accelerated MRI. CoRR (2018)
Aaron Defazio, Léon Bottou
On the Ineffectiveness of Variance Reduced Optimization for Deep Learning. CoRR (2018)
Aaron Defazio, Léon Bottou
Controlling Covariate Shift using Equilibrium Normalization of Weights. CoRR (2018)
Aaron Defazio
On the Curved Geometry of Accelerated Optimization. CoRR (2018)
Aaron Defazio
A Simple Practical Accelerated Method for Finite Sums. CoRR (2016)
Aaron Defazio
A Simple Practical Accelerated Method for Finite Sums. NIPS (2016)
Mark Schmidt, Reza Babanezhad, Mohamed Osama Ahmed, Aaron Defazio, Ann Clifton, Anoop Sarkar
Non-Uniform Stochastic Average Gradient Method for Training Conditional Random Fields. AISTATS (2015)
Aaron Defazio
New Optimisation Methods for Machine Learning. CoRR (2015)
Mark Schmidt, Reza Babanezhad, Mohamed Osama Ahmed, Aaron Defazio, Ann Clifton, Anoop Sarkar
Non-Uniform Stochastic Average Gradient Method for Training Conditional Random Fields. CoRR (2015)
Aaron Defazio, Justin Domke, Tibério S. Caetano
Finito: A faster, permutable incremental gradient method for big data problems. ICML (2014)
Aaron Defazio, Francis R. Bach, Simon Lacoste-Julien
SAGA: A Fast Incremental Gradient Method With Support for Non-Strongly Convex Composite Objectives. CoRR (2014)
Aaron Defazio, Thore Graepel
A Comparison of learning algorithms on the Arcade Learning Environment. CoRR (2014)
Aaron Defazio, Francis R. Bach, Simon Lacoste-Julien
SAGA: A Fast Incremental Gradient Method With Support for Non-Strongly Convex Composite Objectives. NIPS (2014)
Aaron Defazio, Tibério S. Caetano
A Graphical Model Formulation of Collaborative Filtering Neighbourhood Methods with Fast Maximum Entropy Training. ICML (2012)
Aaron Defazio, Tibério S. Caetano
A Convex Formulation for Learning Scale-Free Networks via Submodular Relaxation. NIPS (2012)
Aaron Defazio, Tibério S. Caetano
A Graphical Model Formulation of Collaborative Filtering Neighbourhood Methods with Fast Maximum Entropy Training CoRR (2012)