James Martens

Publication Activity (10 Years)

Years Active: 2010-2024
Publications (10 Years): 28

Top Topics

Gradient Method

Effective Learning

Kronecker Product

Top Venues

Publications

Clare Lyle, Zeyu Zheng, Khimya Khetarpal, James Martens, Hado van Hasselt, Razvan Pascanu, Will Dabney
Normalization and effective learning rates in reinforcement learning. CoRR (2024)
Clare Lyle, Zeyu Zheng, Khimya Khetarpal, Hado van Hasselt, Razvan Pascanu, James Martens, Will Dabney
Disentangling the Causes of Plasticity Loss in Neural Networks. CoRR (2024)
Sheheryar Zaidi, Michael Schaarschmidt, James Martens, Hyunjik Kim, Yee Whye Teh, Alvaro Sanchez-Gonzalez, Peter W. Battaglia, Razvan Pascanu, Jonathan Godwin
Pre-training via Denoising for Molecular Property Prediction. ICLR (2023)
Bobby He, James Martens, Guodong Zhang, Aleksandar Botev, Andrew Brock, Samuel L. Smith, Yee Whye Teh
Deep Transformers without Shortcuts: Modifying Self-attention for Faithful Signal Propagation. CoRR (2023)
Bobby He, James Martens, Guodong Zhang, Aleksandar Botev, Andrew Brock, Samuel L. Smith, Yee Whye Teh
Deep Transformers without Shortcuts: Modifying Self-attention for Faithful Signal Propagation. ICLR (2023)
Sheheryar Zaidi, Michael Schaarschmidt, James Martens, Hyunjik Kim, Yee Whye Teh, Alvaro Sanchez-Gonzalez, Peter W. Battaglia, Razvan Pascanu, Jonathan Godwin
Pre-training via Denoising for Molecular Property Prediction. CoRR (2022)
Guodong Zhang, Aleksandar Botev, James Martens
Deep Learning without Shortcuts: Shaping the Kernel with Tailored Rectifiers. ICLR (2022)
Guodong Zhang, Aleksandar Botev, James Martens
Deep Learning without Shortcuts: Shaping the Kernel with Tailored Rectifiers. CoRR (2022)
James Martens, Andy Ballard, Guillaume Desjardins, Grzegorz Swirszcz, Valentin Dalibard, Jascha Sohl-Dickstein, Samuel S. Schoenholz
Rapid training of deep neural networks without skip connections or normalization layers using Deep Kernel Shaping. CoRR (2021)
James Martens
On the validity of kernel approximations for orthogonally-initialized neural networks. CoRR (2021)
Rahma Mukta, James Martens, Hye-Young Paik, Qinghua Lu, Salil S. Kanhere
Blockchain-based Verifiable Credential Sharing with Selective Disclosure. TrustCom (2020)
James Martens
New Insights and Perspectives on the Natural Gradient Method. J. Mach. Learn. Res. 21 (2020)
Guodong Zhang, James Martens, Roger B. Grosse
Fast Convergence of Natural Gradient Descent for Over-Parameterized Neural Networks. NeurIPS (2019)
Alistair Letcher, David Balduzzi, Sébastien Racanière, James Martens, Jakob N. Foerster, Karl Tuyls, Thore Graepel
Differentiable Game Mechanics. J. Mach. Learn. Res. 20 (2019)
Chongli Qin, James Martens, Sven Gowal, Dilip Krishnan, Krishnamurthy Dvijotham, Alhussein Fawzi, Soham De, Robert Stanforth, Pushmeet Kohli
Adversarial Robustness through Local Linearization. CoRR (2019)
Chongli Qin, James Martens, Sven Gowal, Dilip Krishnan, Krishnamurthy Dvijotham, Alhussein Fawzi, Soham De, Robert Stanforth, Pushmeet Kohli
Adversarial Robustness through Local Linearization. NeurIPS (2019)
Alistair Letcher, David Balduzzi, Sébastien Racanière, James Martens, Jakob N. Foerster, Karl Tuyls, Thore Graepel
Differentiable Game Mechanics. CoRR (2019)
Tim Cooijmans, James Martens
On the Variance of Unbiased Online Recurrent Optimization. CoRR (2019)
Guodong Zhang, Lala Li, Zachary Nado, James Martens, Sushant Sachdeva, George E. Dahl, Christopher J. Shallue, Roger B. Grosse
Which Algorithmic Choices Matter at Which Batch Sizes? Insights From a Noisy Quadratic Model. NeurIPS (2019)
Guodong Zhang, Lala Li, Zachary Nado, James Martens, Sushant Sachdeva, George E. Dahl, Christopher J. Shallue, Roger B. Grosse
Which Algorithmic Choices Matter at Which Batch Sizes? Insights From a Noisy Quadratic Model. CoRR (2019)
Guodong Zhang, James Martens, Roger B. Grosse
Fast Convergence of Natural Gradient Descent for Overparameterized Neural Networks. CoRR (2019)
James Martens, Jimmy Ba, Matt Johnson
Kronecker-factored Curvature Approximations for Recurrent Neural Networks. ICLR (Poster) (2018)
David Balduzzi, Sébastien Racanière, James Martens, Jakob N. Foerster, Karl Tuyls, Thore Graepel
The Mechanics of n-Player Differentiable Games. CoRR (2018)
David Balduzzi, Sébastien Racanière, James Martens, Jakob N. Foerster, Karl Tuyls, Thore Graepel
The Mechanics of n-Player Differentiable Games. ICML (2018)
Zachary Nado, Jasper Snoek, Roger B. Grosse, David Duvenaud, Bowen Xu, James Martens
Stochastic Gradient Langevin dynamics that Exploit Neural Network Structure. ICLR (Workshop) (2018)
Jimmy Ba, Roger B. Grosse, James Martens
Distributed Second-Order Optimization using Kronecker-Factored Approximations. ICLR (Poster) (2017)
Roger B. Grosse, James Martens
A Kronecker-factored approximate Fisher matrix for convolution layers. CoRR (2016)
Roger B. Grosse, James Martens
A Kronecker-factored approximate Fisher matrix for convolution layers. ICML (2016)
James Martens, Roger B. Grosse
Optimizing Neural Networks with Kronecker-factored Approximate Curvature. CoRR (2015)
Arvind Neelakantan, Luke Vilnis, Quoc V. Le, Ilya Sutskever, Lukasz Kaiser, Karol Kurach, James Martens
Adding Gradient Noise Improves Learning for Very Deep Networks. CoRR (2015)
James Martens, Roger B. Grosse
Optimizing Neural Networks with Kronecker-factored Approximate Curvature. ICML (2015)
James Martens, Venkatesh Medabalimi
On the Expressive Efficiency of Sum Product Networks. CoRR (2014)
James Martens
New perspectives on the natural gradient method. CoRR (2014)
Ilya Sutskever, James Martens, George E. Dahl, Geoffrey E. Hinton
On the importance of initialization and momentum in deep learning. ICML (3) (2013)
James Martens, Arkadev Chattopadhyay, Toniann Pitassi, Richard S. Zemel
On the Expressive Power of Restricted Boltzmann Machines. NIPS (2013)
James Martens, Ilya Sutskever, Kevin Swersky
Estimating the Hessian by Back-propagating Curvature. ICML (2012)
Chris Eliasmith, James Martens
Normalization for probabilistic inference with neurons. Biol. Cybern. 104 (4-5) (2011)
Ilya Sutskever, James Martens, Geoffrey E. Hinton
Generating Text with Recurrent Neural Networks. ICML (2011)
James Martens, Ilya Sutskever
Learning Recurrent Neural Networks with Hessian-Free Optimization. ICML (2011)
James Martens
Deep learning via Hessian-free optimization. ICML (2010)
James Martens
Learning the Linear Dynamical System with ASOS. ICML (2010)
James Martens, Ilya Sutskever
Parallelizable Sampling of Markov Random Fields. AISTATS (2010)