Frederik Kunstner

Publication Activity (10 Years)

Years Active: 2018-2024
Publications (10 Years): 16

Top Topics

Order Statistics

Natural Gradient

Extended Abstract

Top Venues

Publications

Frederik Kunstner, Robin Yadav, Alan Milligan, Mark Schmidt, Alberto Bietti
Heavy-Tailed Class Imbalance and Why Adam Outperforms Gradient Descent on Language Models. CoRR (2024)
Frederik Kunstner, Jacques Chen, Jonathan Wilder Lavington, Mark Schmidt
Noise Is Not the Main Factor Behind the Gap Between Sgd and Adam on Transformers, But Sign Descent Might Be. ICLR (2023)
Frederik Kunstner, Victor S. Portella, Mark Schmidt, Nick Harvey
Searching for Optimal Per-Coordinate Step-sizes with Multidimensional Backtracking. CoRR (2023)
Frederik Kunstner, Jacques Chen, Jonathan Wilder Lavington, Mark Schmidt
Noise Is Not the Main Factor Behind the Gap Between SGD and Adam on Transformers, but Sign Descent Might Be. CoRR (2023)
Frederik Kunstner, Victor Sanches Portella, Mark Schmidt, Nicholas J. A. Harvey
Searching for Optimal Per-Coordinate Step-sizes with Multidimensional Backtracking. NeurIPS (2023)
Frederik Kunstner, Raunak Kumar, Mark Schmidt
Homeomorphic-Invariance of EM: Non-Asymptotic Convergence in KL Divergence for Exponential Families via Mirror Descent (Extended Abstract). IJCAI (2022)
Rémi Le Priol, Frederik Kunstner, Damien Scieur, Simon Lacoste-Julien
Convergence Rates for the MAP of an Exponential Family and Stochastic Mirror Descent - an Open Problem. CoRR (2021)
Frederik Kunstner, Raunak Kumar, Mark Schmidt
Homeomorphic-Invariance of EM: Non-Asymptotic Convergence in KL Divergence for Exponential Families via Mirror Descent. AISTATS (2021)
Sharan Vaswani, Frederik Kunstner, Issam H. Laradji, Si Yi Meng, Mark Schmidt, Simon Lacoste-Julien
Adaptive Gradient Methods Converge Faster with Over-Parameterization (and you can do a line-search). CoRR (2020)
Felix Dangel, Frederik Kunstner, Philipp Hennig
BackPACK: Packing more into Backprop. ICLR (2020)
Frederik Kunstner, Raunak Kumar, Mark Schmidt
Homeomorphic-Invariance of EM: Non-Asymptotic Convergence in KL Divergence for Exponential Families via Mirror Descent. CoRR (2020)
Frederik Kunstner, Philipp Hennig, Lukas Balles
Limitations of the empirical Fisher approximation for natural gradient descent. NeurIPS (2019)
Felix Dangel, Frederik Kunstner, Philipp Hennig
BackPACK: Packing more into backprop. CoRR (2019)
Frederik Kunstner, Lukas Balles, Philipp Hennig
Limitations of the Empirical Fisher Approximation. CoRR (2019)
Aaron Mishkin, Frederik Kunstner, Didrik Nielsen, Mark Schmidt, Mohammad Emtiyaz Khan
SLANG: Fast Structured Covariance Approximations for Bayesian Deep Learning with Natural Gradient. NeurIPS (2018)
Aaron Mishkin, Frederik Kunstner, Didrik Nielsen, Mark Schmidt, Mohammad Emtiyaz Khan
SLANG: Fast Structured Covariance Approximations for Bayesian Deep Learning with Natural Gradient. CoRR (2018)