Rae, Jack

12 publications

NeurIPS 2022 An Empirical Analysis of Compute-Optimal Large Language Model Training Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, Thomas Hennigan, Eric Noland, Katherine Millican, George van den Driessche, Bogdan Damoc, Aurelia Guy, Simon Osindero, Karén Simonyan, Erich Elsen, Oriol Vinyals, Jack Rae, Laurent Sifre

ICML 2022 Improving Language Models by Retrieving from Trillions of Tokens Sebastian Borgeaud, Arthur Mensch, Jordan Hoffmann, Trevor Cai, Eliza Rutherford, Katie Millican, George Bm Van Den Driessche, Jean-Baptiste Lespiau, Bogdan Damoc, Aidan Clark, Diego De Las Casas, Aurelia Guy, Jacob Menick, Roman Ring, Tom Hennigan, Saffron Huang, Loren Maggiore, Chris Jones, Albin Cassirer, Andy Brock, Michela Paganini, Geoffrey Irving, Oriol Vinyals, Simon Osindero, Karen Simonyan, Jack Rae, Erich Elsen, Laurent Sifre

ICML 2022 Unified Scaling Laws for Routed Language Models Aidan Clark, Diego De Las Casas, Aurelia Guy, Arthur Mensch, Michela Paganini, Jordan Hoffmann, Bogdan Damoc, Blake Hechtman, Trevor Cai, Sebastian Borgeaud, George Bm Van Den Driessche, Eliza Rutherford, Tom Hennigan, Matthew J Johnson, Albin Cassirer, Chris Jones, Elena Buchatskaya, David Budden, Laurent Sifre, Simon Osindero, Oriol Vinyals, Marc’Aurelio Ranzato, Jack Rae, Erich Elsen, Koray Kavukcuoglu, Karen Simonyan

ICLR 2020 Multiplicative Interactions and Where to Find Them Siddhant M. Jayakumar, Wojciech M. Czarnecki, Jacob Menick, Jonathan Schwarz, Jack Rae, Simon Osindero, Yee Whye Teh, Tim Harley, Razvan Pascanu

ICML 2020 Stabilizing Transformers for Reinforcement Learning Emilio Parisotto, Francis Song, Jack Rae, Razvan Pascanu, Caglar Gulcehre, Siddhant Jayakumar, Max Jaderberg, Raphaël Lopez Kaufman, Aidan Clark, Seb Noury, Matthew Botvinick, Nicolas Heess, Raia Hadsell

NeurIPS 2020 Top-KAST: Top-K Always Sparse Training Siddhant Jayakumar, Razvan Pascanu, Jack Rae, Simon Osindero, Erich Elsen

ICML 2019 Meta-Learning Neural Bloom Filters Jack Rae, Sergey Bartunov, Timothy Lillicrap

NeurIPS 2019 Training Language GANs from Scratch Cyprien de Masson d'Autume, Shakir Mohamed, Mihaela Rosca, Jack Rae

ICML 2018 Fast Parametric Learning with Activation Memorization Jack Rae, Chris Dyer, Peter Dayan, Timothy Lillicrap

NeurIPS 2018 Neural Arithmetic Logic Units Andrew Trask, Felix Hill, Scott E Reed, Jack Rae, Chris Dyer, Phil Blunsom

NeurIPS 2018 Relational Recurrent Neural Networks Adam Santoro, Ryan Faulkner, David Raposo, Jack Rae, Mike Chrzanowski, Theophane Weber, Daan Wierstra, Oriol Vinyals, Razvan Pascanu, Timothy Lillicrap

NeurIPS 2016 Scaling Memory-Augmented Neural Networks with Sparse Reads and Writes Jack Rae, Jonathan J Hunt, Ivo Danihelka, Timothy Harley, Andrew W. Senior, Gregory Wayne, Alex Graves, Timothy Lillicrap