Tanaka, Hidenori

46 publications

ICLR 2025 A Percolation Model of Emergence: Analyzing Transformers Trained on a Formal Language Ekdeep Singh Lubana, Kyogo Kawaguchi, Robert P. Dick, Hidenori Tanaka

ICLR 2025 Competition Dynamics Shape Algorithmic Phases of In-Context Learning Core Francisco Park, Ekdeep Singh Lubana, Hidenori Tanaka

ICML 2025 Dynamical Phases of Short-Term Memory Mechanisms in RNNs Bariscan Kurtkaya, Fatih Dinc, Mert Yuksekgonul, Marta Blanco-Pozo, Ege Cirakman, Mark Schnitzer, Yucel Yemez, Hidenori Tanaka, Peng Yuan, Nina Miolane

ICLR 2025 Forking Paths in Neural Text Generation Eric J Bigelow, Ari Holtzman, Hidenori Tanaka, Tomer Ullman

ICLR 2025 ICLR: In-Context Learning of Representations Core Francisco Park, Andrew Lee, Ekdeep Singh Lubana, Yongyi Yang, Maya Okawa, Kento Nishi, Martin Wattenberg, Hidenori Tanaka

NeurIPS 2025 In-Context Learning Strategies Emerge Rationally Daniel Wurgaft, Ekdeep Singh Lubana, Core Francisco Park, Hidenori Tanaka, Gautam Reddy, Noah Goodman

ICML 2025 Representation Shattering in Transformers: A Synthetic Study with Knowledge Editing Kento Nishi, Rahul Ramesh, Maya Okawa, Mikail Khona, Hidenori Tanaka, Ekdeep Singh Lubana

ICLR 2025 Swing-by Dynamics in Concept Learning and Compositional Generalization Yongyi Yang, Core Francisco Park, Ekdeep Singh Lubana, Maya Okawa, Wei Hu, Hidenori Tanaka

NeurIPSW 2024 A Continuous-Time Analysis of Adaptive Optimization and Normalization Rhys Gould, Hidenori Tanaka

ICML 2024 Compositional Capabilities of Autoregressive Transformers: A Study on Synthetic, Interpretable Tasks Rahul Ramesh, Ekdeep Singh Lubana, Mikail Khona, Robert P. Dick, Hidenori Tanaka

NeurIPSW 2024 Continuous-Time Analysis of Adaptive Optimization and Normalization Rhys Gould, Hidenori Tanaka

NeurIPSW 2024 Dynamics of Concept Learning and Compositional Generalization Yongyi Yang, Core Francisco Park, Ekdeep Singh Lubana, Maya Okawa, Wei Hu, Hidenori Tanaka

NeurIPS 2024 Emergence of Hidden Capabilities: Exploring Learning Dynamics in Concept Space Core Francisco Park, Maya Okawa, Andrew Lee, Hidenori Tanaka, Ekdeep Singh Lubana

NeurIPSW 2024 Emergence of Hierarchical Emotion Representations in Large Language Models Bo Zhao, Maya Okawa, Eric J Bigelow, Rose Yu, Tomer Ullman, Hidenori Tanaka

ICMLW 2024 Hidden Learning Dynamics of Capability Before Behavior in Diffusion Models Core Francisco Park, Maya Okawa, Andrew Lee, Ekdeep Singh Lubana, Hidenori Tanaka

ICLR 2024 In-Context Learning Dynamics with Random Binary Sequences Eric J Bigelow, Ekdeep Singh Lubana, Robert P. Dick, Hidenori Tanaka, Tomer Ullman

ICLR 2024 Mechanistically Analyzing the Effects of Fine-Tuning on Procedurally Defined Tasks Samyak Jain, Robert Kirk, Ekdeep Singh Lubana, Robert P. Dick, Hidenori Tanaka, Tim Rocktäschel, Edward Grefenstette, David Krueger

ICLRW 2024 Mechanistically Analyzing the Effects of Fine-Tuning on Procedurally Defined Tasks Samyak Jain, Robert Kirk, Ekdeep Singh Lubana, Robert P. Dick, Hidenori Tanaka, Tim Rocktäschel, Edward Grefenstette, David Krueger

NeurIPSW 2024 Structured Identity Mapping Learning as a Model for Compositional Generalization in Generative Models Yongyi Yang, Core Francisco Park, Ekdeep Singh Lubana, Maya Okawa, Wei Hu, Hidenori Tanaka

NeurIPSW 2024 Structured In-Context Task Representations Core Francisco Park, Andrew Lee, Ekdeep Singh Lubana, Kento Nishi, Maya Okawa, Hidenori Tanaka

ICMLW 2024 The Concept Percolation Hypothesis: Analyzing the Emergence of Capabilities in Neural Networks Trained on Formal Grammars Ekdeep Singh Lubana, Kyogo Kawaguchi, Robert P. Dick, Hidenori Tanaka

ICML 2024 Towards an Understanding of Stepwise Inference in Transformers: A Synthetic Graph Navigation Model Mikail Khona, Maya Okawa, Jan Hula, Rahul Ramesh, Kento Nishi, Robert P. Dick, Ekdeep Singh Lubana, Hidenori Tanaka

NeurIPSW 2024 Understanding the Transient Nature of In-Context Learning: The Window of Generalization Core Francisco Park, Ekdeep Singh Lubana, Hidenori Tanaka

ICMLW 2024 Why Do Recurrent Neural Networks Suddenly Learn? Bifurcation Mechanisms in Neuro-Inspired Short-Term Memory Tasks Udith Haputhanthri, Liam Storan, Yiqi Jiang, Adam Shai, Hakki Orhun Akengin, Mark Schnitzer, Fatih Dinc, Hidenori Tanaka

NeurIPS 2023 CORNN: Convex Optimization of Recurrent Neural Networks for Rapid Inference of Neural Dynamics Fatih Dinc, Adam Shai, Mark Schnitzer, Hidenori Tanaka

NeurIPS 2023 Compositional Abilities Emerge Multiplicatively: Exploring Diffusion Models on a Synthetic Task Maya Okawa, Ekdeep S Lubana, Robert Dick, Hidenori Tanaka

ICMLW 2023 Compositional Abilities Emerge Multiplicatively: Exploring Diffusion Models on a Synthetic Task Maya Okawa, Ekdeep Singh Lubana, Robert P. Dick, Hidenori Tanaka

NeurIPSW 2023 Enhanced Cue Associated Memory in Temporally Consistent Recurrent Neural Networks Udith Haputhanthri, Liam Storan, Adam Shai, Surya Ganguli, Mark Schnitzer, Hidenori Tanaka, Fatih Dinc

NeurIPSW 2023 How Capable Can a Transformer Become? a Study on Synthetic, Interpretable Tasks Rahul Ramesh, Mikail Khona, Robert P. Dick, Hidenori Tanaka, Ekdeep Singh Lubana

NeurIPSW 2023 How Capable Can a Transformer Become? a Study on Synthetic, Interpretable Tasks Rahul Ramesh, Mikail Khona, Robert P. Dick, Hidenori Tanaka, Ekdeep Singh Lubana

NeurIPSW 2023 How Does Fine-Tuning Affect Your Model? Mechanistic Analysis on Procedural Tasks Samyak Jain, Robert Kirk, Ekdeep Singh Lubana, Robert P. Dick, Hidenori Tanaka, Tim Rocktäschel, Edward Grefenstette, David Krueger

NeurIPSW 2023 How Does Fine-Tuning Affect Your Model? Mechanistic Analysis on Procedural Tasks Samyak Jain, Robert Kirk, Ekdeep Singh Lubana, Robert P. Dick, Hidenori Tanaka, Tim Rocktäschel, Edward Grefenstette, David Krueger

ICML 2023 Mechanistic Mode Connectivity Ekdeep Singh Lubana, Eric J Bigelow, Robert P. Dick, David Krueger, Hidenori Tanaka

NeurIPSW 2023 Stepwise Inference in Transformers: Exploring a Synthetic Graph Navigation Task Mikail Khona, Maya Okawa, Rahul Ramesh, Kento Nishi, Robert P. Dick, Ekdeep Singh Lubana, Hidenori Tanaka

NeurIPSW 2023 Subjective Randomness and In-Context Learning Eric J Bigelow, Ekdeep Singh Lubana, Robert P. Dick, Hidenori Tanaka, Tomer Ullman

ICLR 2023 What Shapes the Loss Landscape of Self Supervised Learning? Liu Ziyin, Ekdeep Singh Lubana, Masahito Ueda, Hidenori Tanaka

NeurIPSW 2022 A Mechanistic Lens on Mode Connectivity Ekdeep Singh Lubana, Eric J Bigelow, Robert P. Dick, David Krueger, Hidenori Tanaka

NeurIPSW 2022 Geometric Considerations for Normalization Layers in Equivariant Neural Networks Max Shirokawa Aalto, Ekdeep Singh Lubana, Hidenori Tanaka

NeurIPSW 2022 Mechanistic Lens on Mode Connectivity Ekdeep Singh Lubana, Eric J Bigelow, Robert P. Dick, David Krueger, Hidenori Tanaka

NeurIPSW 2022 On Rotational Symmetry in the Loss Landscape of Self-Supervised Learning Liu Ziyin, Ekdeep Singh Lubana, Masahito Ueda, Hidenori Tanaka

NeurIPS 2021 Beyond BatchNorm: Towards a Unified Understanding of Normalization in Deep Learning Ekdeep S Lubana, Robert Dick, Hidenori Tanaka

ICLR 2021 Neural Mechanics: Symmetry and Broken Conservation Laws in Deep Learning Dynamics Daniel Kunin, Javier Sagastuy-Brena, Surya Ganguli, Daniel LK Yamins, Hidenori Tanaka

NeurIPS 2021 Noether’s Learning Dynamics: Role of Symmetry Breaking in Neural Networks Hidenori Tanaka, Daniel Kunin

NeurIPS 2020 Pruning Neural Networks Without Any Data by Iteratively Conserving Synaptic Flow Hidenori Tanaka, Daniel Kunin, Daniel L Yamins, Surya Ganguli

NeurIPS 2019 From Deep Learning to Mechanistic Understanding in Neuroscience: The Structure of Retinal Prediction Hidenori Tanaka, Aran Nayebi, Niru Maheswaranathan, Lane McIntosh, Stephen Baccus, Surya Ganguli

NeurIPSW 2019 Revealing Computational Mechanisms of Retinal Prediction via Model Reduction Hidenori Tanaka, Aran Nayebi, Niru Maheswaranathan, Lane McIntosh, Stephen A. Baccus, Surya Ganguli