Lubana, Ekdeep Singh

49 publications

ICLR 2025 A Percolation Model of Emergence: Analyzing Transformers Trained on a Formal Language Ekdeep Singh Lubana, Kyogo Kawaguchi, Robert P. Dick, Hidenori Tanaka

ICML 2025 Archetypal SAE: Adaptive and Stable Dictionary Learning for Concept Extraction in Large Vision Models Thomas Fel, Ekdeep Singh Lubana, Jacob S. Prince, Matthew Kowal, Victor Boutin, Isabel Papadimitriou, Binxu Wang, Martin Wattenberg, Demba E. Ba, Talia Konkle

ICLR 2025 Competition Dynamics Shape Algorithmic Phases of In-Context Learning Core Francisco Park, Ekdeep Singh Lubana, Hidenori Tanaka

NeurIPS 2025 Detecting High-Stakes Interactions with Activation Probes Alex McKenzie, Urja Pawar, Phil Blandfort, William Bankes, David Krueger, Ekdeep Singh Lubana, Dmitrii Krasheninnikov

NeurIPS 2025 From Flat to Hierarchical: Extracting Sparse Representations with Matching Pursuit Valérie Costa, Thomas Fel, Ekdeep Singh Lubana, Bahareh Tolooshams, Demba E. Ba

ICLR 2025 ICLR: In-Context Learning of Representations Core Francisco Park, Andrew Lee, Ekdeep Singh Lubana, Yongyi Yang, Maya Okawa, Kento Nishi, Martin Wattenberg, Hidenori Tanaka

NeurIPS 2025 In-Context Learning Strategies Emerge Rationally Daniel Wurgaft, Ekdeep Singh Lubana, Core Francisco Park, Hidenori Tanaka, Gautam Reddy, Noah Goodman

NeurIPS 2025 Projecting Assumptions: The Duality Between Sparse Autoencoders and Concept Geometry Sai Sumedh R. Hindupur, Ekdeep Singh Lubana, Thomas Fel, Demba E. Ba

ICML 2025 Representation Shattering in Transformers: A Synthetic Study with Knowledge Editing Kento Nishi, Rahul Ramesh, Maya Okawa, Mikail Khona, Hidenori Tanaka, Ekdeep Singh Lubana

ICLR 2025 Swing-by Dynamics in Concept Learning and Compositional Generalization Yongyi Yang, Core Francisco Park, Ekdeep Singh Lubana, Maya Okawa, Wei Hu, Hidenori Tanaka

NeurIPS 2024 Abrupt Learning in Transformers: A Case Study on Matrix Completion Pulkit Gopalani, Ekdeep Singh Lubana, Wei Hu

NeurIPSW 2024 Analyzing (In)Abilities of SAEs via Formal Languages Abhinav Menon, Manish Shrivastava, Ekdeep Singh Lubana, David Krueger

ICML 2024 Compositional Capabilities of Autoregressive Transformers: A Study on Synthetic, Interpretable Tasks Rahul Ramesh, Ekdeep Singh Lubana, Mikail Khona, Robert P. Dick, Hidenori Tanaka

NeurIPSW 2024 Dynamics of Concept Learning and Compositional Generalization Yongyi Yang, Core Francisco Park, Ekdeep Singh Lubana, Maya Okawa, Wei Hu, Hidenori Tanaka

NeurIPS 2024 Emergence of Hidden Capabilities: Exploring Learning Dynamics in Concept Space Core Francisco Park, Maya Okawa, Andrew Lee, Hidenori Tanaka, Ekdeep Singh Lubana

TMLR 2024 Foundational Challenges in Assuring Alignment and Safety of Large Language Models Usman Anwar, Abulhair Saparov, Javier Rando, Daniel Paleka, Miles Turpin, Peter Hase, Ekdeep Singh Lubana, Erik Jenner, Stephen Casper, Oliver Sourbut, Benjamin L. Edelman, Zhaowei Zhang, Mario Günther, Anton Korinek, Jose Hernandez-Orallo, Lewis Hammond, Eric J Bigelow, Alexander Pan, Lauro Langosco, Tomasz Korbak, Heidi Chenyu Zhang, Ruiqi Zhong, Sean O hEigeartaigh, Gabriel Recchia, Giulio Corsi, Alan Chan, Markus Anderljung, Lilian Edwards, Aleksandar Petrov, Christian Schroeder de Witt, Sumeet Ramesh Motwani, Yoshua Bengio, Danqi Chen, Philip Torr, Samuel Albanie, Tegan Maharaj, Jakob Nicolaus Foerster, Florian Tramèr, He He, Atoosa Kasirzadeh, Yejin Choi, David Krueger

ICMLW 2024 Hidden Learning Dynamics of Capability Before Behavior in Diffusion Models Core Francisco Park, Maya Okawa, Andrew Lee, Ekdeep Singh Lubana, Hidenori Tanaka

ICMLW 2024 How Do Transformers Fill in the Blanks? a Case Study on Matrix Completion Pulkit Gopalani, Ekdeep Singh Lubana, Wei Hu

ICLR 2024 In-Context Learning Dynamics with Random Binary Sequences Eric J Bigelow, Ekdeep Singh Lubana, Robert P. Dick, Hidenori Tanaka, Tomer Ullman

ICLR 2024 Mechanistically Analyzing the Effects of Fine-Tuning on Procedurally Defined Tasks Samyak Jain, Robert Kirk, Ekdeep Singh Lubana, Robert P. Dick, Hidenori Tanaka, Tim Rocktäschel, Edward Grefenstette, David Krueger

ICLRW 2024 Mechanistically Analyzing the Effects of Fine-Tuning on Procedurally Defined Tasks Samyak Jain, Robert Kirk, Ekdeep Singh Lubana, Robert P. Dick, Hidenori Tanaka, Tim Rocktäschel, Edward Grefenstette, David Krueger

NeurIPSW 2024 Structured Identity Mapping Learning as a Model for Compositional Generalization in Generative Models Yongyi Yang, Core Francisco Park, Ekdeep Singh Lubana, Maya Okawa, Wei Hu, Hidenori Tanaka

NeurIPSW 2024 Structured In-Context Task Representations Core Francisco Park, Andrew Lee, Ekdeep Singh Lubana, Kento Nishi, Maya Okawa, Hidenori Tanaka

ICMLW 2024 The Concept Percolation Hypothesis: Analyzing the Emergence of Capabilities in Neural Networks Trained on Formal Grammars Ekdeep Singh Lubana, Kyogo Kawaguchi, Robert P. Dick, Hidenori Tanaka

NeurIPSW 2024 Towards Reliable Evaluation of Behavior Steering Interventions in LLMs Itamar Pres, Laura Ruis, Ekdeep Singh Lubana, David Krueger

ICML 2024 Towards an Understanding of Stepwise Inference in Transformers: A Synthetic Graph Navigation Model Mikail Khona, Maya Okawa, Jan Hula, Rahul Ramesh, Kento Nishi, Robert P. Dick, Ekdeep Singh Lubana, Hidenori Tanaka

NeurIPSW 2024 Understanding the Transient Nature of In-Context Learning: The Window of Generalization Core Francisco Park, Ekdeep Singh Lubana, Hidenori Tanaka

NeurIPS 2024 What Makes and Breaks Safety Fine-Tuning? a Mechanistic Study Samyak Jain, Ekdeep Singh Lubana, Kemal Oksuz, Tom Joy, Philip H.S. Torr, Amartya Sanyal, Puneet K. Dokania

ICMLW 2024 What Makes and Breaks Safety Fine-Tuning? a Mechanistic Study Samyak Jain, Ekdeep Singh Lubana, Kemal Oksuz, Tom Joy, Philip Torr, Amartya Sanyal, Puneet K. Dokania

ICMLW 2023 Compositional Abilities Emerge Multiplicatively: Exploring Diffusion Models on a Synthetic Task Maya Okawa, Ekdeep Singh Lubana, Robert P. Dick, Hidenori Tanaka

NeurIPSW 2023 FoMo Rewards: Can We Cast Foundation Models as Reward Functions? Ekdeep Singh Lubana, Johann Brehmer, Pim De Haan, Taco Cohen

NeurIPSW 2023 How Capable Can a Transformer Become? a Study on Synthetic, Interpretable Tasks Rahul Ramesh, Mikail Khona, Robert P. Dick, Hidenori Tanaka, Ekdeep Singh Lubana

NeurIPSW 2023 How Does Fine-Tuning Affect Your Model? Mechanistic Analysis on Procedural Tasks Samyak Jain, Robert Kirk, Ekdeep Singh Lubana, Robert P. Dick, Hidenori Tanaka, Tim Rocktäschel, Edward Grefenstette, David Krueger

ICML 2023 Mechanistic Mode Connectivity Ekdeep Singh Lubana, Eric J Bigelow, Robert P. Dick, David Krueger, Hidenori Tanaka

NeurIPSW 2023 Stepwise Inference in Transformers: Exploring a Synthetic Graph Navigation Task Mikail Khona, Maya Okawa, Rahul Ramesh, Kento Nishi, Robert P. Dick, Ekdeep Singh Lubana, Hidenori Tanaka

NeurIPSW 2023 Subjective Randomness and In-Context Learning Eric J Bigelow, Ekdeep Singh Lubana, Robert P. Dick, Hidenori Tanaka, Tomer Ullman

NeurIPSW 2023 What Mechanisms Does Knowledge Distillation Distill? Cindy Wu, Ekdeep Singh Lubana, Bruno Kacper Mlodozeniec, Robert Kirk, David Krueger

ICLR 2023 What Shapes the Loss Landscape of Self Supervised Learning? Liu Ziyin, Ekdeep Singh Lubana, Masahito Ueda, Hidenori Tanaka

NeurIPSW 2022 A Mechanistic Lens on Mode Connectivity Ekdeep Singh Lubana, Eric J Bigelow, Robert P. Dick, David Krueger, Hidenori Tanaka

NeurIPSW 2022 Geometric Considerations for Normalization Layers in Equivariant Neural Networks Max Shirokawa Aalto, Ekdeep Singh Lubana, Hidenori Tanaka

CoLLAs 2022 How Do Quadratic Regularizers Prevent Catastrophic Forgetting: The Role of Interpolation Ekdeep Singh Lubana, Puja Trivedi, Danai Koutra, Robert Dick

NeurIPSW 2022 Mechanistic Lens on Mode Connectivity Ekdeep Singh Lubana, Eric J Bigelow, Robert P. Dick, David Krueger, Hidenori Tanaka

NeurIPSW 2022 On Rotational Symmetry in the Loss Landscape of Self-Supervised Learning Liu Ziyin, Ekdeep Singh Lubana, Masahito Ueda, Hidenori Tanaka

ICLR 2021 A Gradient Flow Framework for Analyzing Network Pruning Ekdeep Singh Lubana, Robert P. Dick

CVPRW 2020 Intelligent Scene Caching to Improve Accuracy for Energy-Constrained Embedded Vision Benjamin Simpson, Ekdeep Singh Lubana, Yuchen Liu, Robert P. Dick