Johannes von Oswald

Publication Activity (10 Years)

Years Active: 2019-2024
Publications (10 Years): 26

Top Topics

Decision Stumps

Ensemble Selection

Recurrent Neural Networks

Top Venues

Publications

Max Vladymyrov, Johannes von Oswald, Mark Sandler, Rong Ge
Linear Transformers are Versatile In-Context Learners. CoRR (2024)
Seijin Kobayashi, Simon Schug, Yassir Akram, Florian Redhardt, Johannes von Oswald, Razvan Pascanu, Guillaume Lajoie, João Sacramento
When can transformers compositionally generalize in-context? CoRR (2024)
Maciej Pióro, Maciej Wolczyk, Razvan Pascanu, Johannes von Oswald, João Sacramento
State Soup: In-Context Skill Learning, Retrieval and Mixing. CoRR (2024)
Simon Schug, Seijin Kobayashi, Yassir Akram, Maciej Wolczyk, Alexandra Proca, Johannes von Oswald, Razvan Pascanu, João Sacramento, Angelika Steger
Discovering modular solutions that generalize compositionally. ICLR (2024)
Simon Schug, Seijin Kobayashi, Yassir Akram, Maciej Wolczyk, Alexandra Proca, Johannes von Oswald, Razvan Pascanu, João Sacramento, Angelika Steger
Discovering modular solutions that generalize compositionally. CoRR (2023)
Nicolas Zucchet, Seijin Kobayashi, Yassir Akram, Johannes von Oswald, Maxime Larcher, Angelika Steger, João Sacramento
Gated recurrent neural networks discover attention. CoRR (2023)
Johannes von Oswald, Eyvind Niklasson, Maximilian Schlegel, Seijin Kobayashi, Nicolas Zucchet, Nino Scherrer, Nolan Miller, Mark Sandler, Blaise Agüera y Arcas, Max Vladymyrov, Razvan Pascanu, João Sacramento
Uncovering mesa-optimization algorithms in Transformers. CoRR (2023)
Johannes von Oswald, Eyvind Niklasson, Ettore Randazzo, João Sacramento, Alexander Mordvintsev, Andrey Zhmoginov, Max Vladymyrov
Transformers Learn In-Context by Gradient Descent. ICML (2023)
Alexander Meulemans, Nicolas Zucchet, Seijin Kobayashi, Johannes von Oswald, João Sacramento
The least-control principle for local learning at equilibrium. NeurIPS (2022)
Seijin Kobayashi, Pau Vilimelis Aceituno, Johannes von Oswald
Disentangling the Predictive Variance of Deep Ensembles through the Neural Tangent Kernel. NeurIPS (2022)
Seijin Kobayashi, Pau Vilimelis Aceituno, Johannes von Oswald
Disentangling the Predictive Variance of Deep Ensembles through the Neural Tangent Kernel. CoRR (2022)
Alexander Meulemans, Nicolas Zucchet, Seijin Kobayashi, Johannes von Oswald, João Sacramento
The least-control principle for learning at equilibrium. CoRR (2022)
Johannes von Oswald, Eyvind Niklasson, Ettore Randazzo, João Sacramento, Alexander Mordvintsev, Andrey Zhmoginov, Max Vladymyrov
Transformers learn in-context by gradient descent. CoRR (2022)
Frederik Benzing, Simon Schug, Robert Meier, Johannes von Oswald, Yassir Akram, Nicolas Zucchet, Laurence Aitchison, Angelika Steger
Random initialisations performing above chance and how to find them. CoRR (2022)
Nicolas Zucchet, Simon Schug, Johannes von Oswald, Dominic Zhao, João Sacramento
A contrastive rule for meta-learning. NeurIPS (2022)
Christian Henning, Maria R. Cervera, Francesco D'Angelo, Johannes von Oswald, Regina Traber, Benjamin Ehret, Seijin Kobayashi, João Sacramento, Benjamin F. Grewe
Posterior Meta-Replay for Continual Learning. CoRR (2021)
Johannes von Oswald, Seijin Kobayashi, João Sacramento, Alexander Meulemans, Christian Henning, Benjamin F. Grewe
Neural networks with late-phase weights. ICLR (2021)
Christian Henning, Maria R. Cervera, Francesco D'Angelo, Johannes von Oswald, Regina Traber, Benjamin Ehret, Seijin Kobayashi, Benjamin F. Grewe, João Sacramento
Posterior Meta-Replay for Continual Learning. NeurIPS (2021)
Johannes von Oswald, Dominic Zhao, Seijin Kobayashi, Simon Schug, Massimo Caccia, Nicolas Zucchet, João Sacramento
Learning where to learn: Gradient sparsity in meta and continual learning. NeurIPS (2021)
Nicolas Zucchet, Simon Schug, Johannes von Oswald, Dominic Zhao, João Sacramento
A contrastive rule for meta-learning. CoRR (2021)
Johannes von Oswald, Dominic Zhao, Seijin Kobayashi, Simon Schug, Massimo Caccia, Nicolas Zucchet, João Sacramento
Learning where to learn: Gradient sparsity in meta and continual learning. CoRR (2021)
Benjamin Ehret, Christian Henning, Maria R. Cervera, Alexander Meulemans, Johannes von Oswald, Benjamin F. Grewe
Continual learning in recurrent neural networks. ICLR (2021)
Benjamin Ehret, Christian Henning, Maria R. Cervera, Alexander Meulemans, Johannes von Oswald, Benjamin F. Grewe
Continual Learning in Recurrent Neural Networks with Hypernetworks. CoRR (2020)
Johannes von Oswald, Christian Henning, João Sacramento, Benjamin F. Grewe
Continual learning with hypernetworks. ICLR (2020)
João Sacramento, Johannes von Oswald, Seijin Kobayashi, Christian Henning, Benjamin F. Grewe
Economical ensembles with hypernetworks. CoRR (2020)
Johannes von Oswald, Christian Henning, João Sacramento, Benjamin F. Grewe
Continual learning with hypernetworks. CoRR (2019)