Irie, Kazuki

24 publications

TMLR 2026 Fast Weight Programming and Linear Transformers: From Machine Learning to Neurobiology Kazuki Irie, Samuel J. Gershman

NeurIPS 2025 Blending Complementary Memory Systems in Hybrid Quadratic-Linear Transformers Kazuki Irie, Morris Yau, Samuel J. Gershman

TMLR 2025 Metalearning Continual Learning Algorithms Kazuki Irie, Róbert Csordás, Jürgen Schmidhuber

NeurIPS 2024 Dissecting the Interplay of Attention Paths in a Statistical Mechanics Theory of Transformers Lorenzo Tiberi, Francesca Mignacco, Kazuki Irie, Haim Sompolinsky

ICLR 2024 Exploring the Promise and Limits of Real-Time Recurrent Learning Kazuki Irie, Anand Gopalakrishnan, Jürgen Schmidhuber

NeurIPS 2024 MoEUT: Mixture-of-Experts Universal Transformers Róbert Csordás, Kazuki Irie, Jürgen Schmidhuber, Christopher Potts, Christopher D. Manning

NeurIPSW 2024 Neural Representational Geometry of Concepts in Large Language Models Linden Schrage, Kazuki Irie, Haim Sompolinsky

NeurIPS 2024 SwitchHead: Accelerating Transformers with Mixture-of-Experts Attention Róbert Csordás, Piotr Piękos, Kazuki Irie, Jürgen Schmidhuber

ICLRW 2023 Accelerating Neural Self-Improvement via Bootstrapping Kazuki Irie, Jürgen Schmidhuber

NeurIPS 2023 Contrastive Training of Complex-Valued Autoencoders for Object Discovery Aleksandar Stanić, Anand Gopalakrishnan, Kazuki Irie, Jürgen Schmidhuber

ICLR 2023 Images as Weight Matrices: Sequential Image Generation Through Synaptic Learning Rules Kazuki Irie, Jürgen Schmidhuber

NeurIPSW 2023 Mindstorms in Natural Language-Based Societies of Mind Mingchen Zhuge, Haozhe Liu, Francesco Faccio, Dylan R. Ashley, Róbert Csordás, Anand Gopalakrishnan, Abdullah Hamdi, Hasan Abed Al Kader Hammoud, Vincent Herrmann, Kazuki Irie, Louis Kirsch, Bing Li, Guohao Li, Shuming Liu, Jinjie Mai, Piotr Piękos, Aditya Ramesh, Imanol Schlag, Weimin Shi, Aleksandar Stanić, Wenyi Wang, Yuhui Wang, Mengmeng Xu, Deng-Ping Fan, Bernard Ghanem, Jürgen Schmidhuber

ICMLW 2023 Topological Neural Discrete Representation Learning À La Kohonen Kazuki Irie, Róbert Csordás, Jürgen Schmidhuber

ICML 2022 A Modern Self-Referential Weight Matrix That Learns to Modify Itself Kazuki Irie, Imanol Schlag, Róbert Csordás, Jürgen Schmidhuber

NeurIPS 2022 Neural Differential Equations for Learning to Program Neural Nets Through Continuous Learning Rules Kazuki Irie, Francesco Faccio, Jürgen Schmidhuber

ICML 2022 The Dual Form of Neural Networks Revisited: Connecting Test Time Predictions to Training Patterns via Spotlights of Attention Kazuki Irie, Róbert Csordás, Jürgen Schmidhuber

ICLR 2022 The Neural Data Router: Adaptive Control Flow in Transformers Improves Systematic Generalization Róbert Csordás, Kazuki Irie, Jürgen Schmidhuber

NeurIPSW 2021 A Modern Self-Referential Weight Matrix That Learns to Modify Itself Kazuki Irie, Imanol Schlag, Róbert Csordás, Jürgen Schmidhuber

NeurIPS 2021 Going Beyond Linear Transformers with Recurrent Fast Weight Programmers Kazuki Irie, Imanol Schlag, Róbert Csordás, Jürgen Schmidhuber

NeurIPSW 2021 Improving Baselines in the Wild Kazuki Irie, Imanol Schlag, Róbert Csordás, Jürgen Schmidhuber

NeurIPSW 2021 Learning Adaptive Control Flow in Transformers for Improved Systematic Generalization Róbert Csordás, Kazuki Irie, Jürgen Schmidhuber

ICML 2021 Linear Transformers Are Secretly Fast Weight Programmers Imanol Schlag, Kazuki Irie, Jürgen Schmidhuber

ICLRW 2021 Training and Generating Neural Networks in Compressed Weight Space Kazuki Irie, Jürgen Schmidhuber

NeurIPSW 2021 Unsupervised Learning of Temporal Abstractions Using Slot-Based Transformers Anand Gopalakrishnan, Kazuki Irie, Jürgen Schmidhuber, Sjoerd van Steenkiste