Martens, James

24 publications

CoLLAs 2024 Disentangling the Causes of Plasticity Loss in Neural Networks Clare Lyle, Zeyu Zheng, Khimya Khetarpal, Hado van Hasselt, Razvan Pascanu, James Martens, Will Dabney

NeurIPS 2024 Normalization and Effective Learning Rates in Reinforcement Learning Clare Lyle, Zeyu Zheng, Khimya Khetarpal, James Martens, Hado van Hasselt, Razvan Pascanu, Will Dabney

ICLR 2023 Deep Transformers Without Shortcuts: Modifying Self-Attention for Faithful Signal Propagation Bobby He, James Martens, Guodong Zhang, Aleksandar Botev, Andrew Brock, Samuel L Smith, Yee Whye Teh

ICLR 2023 Pre-Training via Denoising for Molecular Property Prediction Sheheryar Zaidi, Michael Schaarschmidt, James Martens, Hyunjik Kim, Yee Whye Teh, Alvaro Sanchez-Gonzalez, Peter Battaglia, Razvan Pascanu, Jonathan Godwin

ICLR 2022 Deep Learning Without Shortcuts: Shaping the Kernel with Tailored Rectifiers Guodong Zhang, Aleksandar Botev, James Martens

NeurIPSW 2022 Pre-Training via Denoising for Molecular Property Prediction Sheheryar Zaidi, Michael Schaarschmidt, James Martens, Hyunjik Kim, Yee Whye Teh, Alvaro Sanchez-Gonzalez, Peter Battaglia, Razvan Pascanu, Jonathan Godwin

JMLR 2020 New Insights and Perspectives on the Natural Gradient Method James Martens

NeurIPS 2019 Adversarial Robustness Through Local Linearization Chongli Qin, James Martens, Sven Gowal, Dilip Krishnan, Krishnamurthy Dvijotham, Alhussein Fawzi, Soham De, Robert Stanforth, Pushmeet Kohli

JMLR 2019 Differentiable Game Mechanics Alistair Letcher, David Balduzzi, Sébastien Racanière, James Martens, Jakob Foerster, Karl Tuyls, Thore Graepel

NeurIPS 2019 Fast Convergence of Natural Gradient Descent for Over-Parameterized Neural Networks Guodong Zhang, James Martens, Roger B Grosse

NeurIPS 2019 Which Algorithmic Choices Matter at Which Batch Sizes? Insights from a Noisy Quadratic Model Guodong Zhang, Lala Li, Zachary Nado, James Martens, Sushant Sachdeva, George Dahl, Chris Shallue, Roger B Grosse

ICLR 2018 Kronecker-Factored Curvature Approximations for Recurrent Neural Networks James Martens, Jimmy Ba, Matt Johnson

ICML 2018 The Mechanics of N-Player Differentiable Games David Balduzzi, Sebastien Racaniere, James Martens, Jakob Foerster, Karl Tuyls, Thore Graepel

ICLR 2017 Distributed Second-Order Optimization Using Kronecker-Factored Approximations Jimmy Ba, Roger B. Grosse, James Martens

ICML 2016 A Kronecker-Factored Approximate Fisher Matrix for Convolution Layers Roger Grosse, James Martens

ICML 2015 Optimizing Neural Networks with Kronecker-Factored Approximate Curvature James Martens, Roger Grosse

ICML 2013 On the Importance of Initialization and Momentum in Deep Learning Ilya Sutskever, James Martens, George Dahl, Geoffrey Hinton

NeurIPS 2013 On the Representational Efficiency of Restricted Boltzmann Machines James Martens, Arkadev Chattopadhya, Toni Pitassi, Richard Zemel

ICML 2012 Estimating the Hessian by Back-Propagating Curvature James Martens, Ilya Sutskever, Kevin Swersky

ICML 2011 Generating Text with Recurrent Neural Networks Ilya Sutskever, James Martens, Geoffrey E. Hinton

ICML 2011 Learning Recurrent Neural Networks with Hessian-Free Optimization James Martens, Ilya Sutskever

ICML 2010 Deep Learning via Hessian-Free Optimization James Martens

ICML 2010 Learning the Linear Dynamical System with ASOS James Martens

AISTATS 2010 Parallelizable Sampling of Markov Random Fields James Martens, Ilya Sutskever