Ganguli, Surya

61 publications

NeurIPS 2025 Alternating Gradient Flows: A Theory of Feature Learning in Two-Layer Neural Networks Daniel Kunin, Giovanni Luca Marchetti, Feng Chen, Dhruva Karkada, James B Simon, Michael R DeWeese, Surya Ganguli, Nina Miolane

ICML 2025 An Analytic Theory of Creativity in Convolutional Diffusion Models Mason Kamb, Surya Ganguli

ICML 2025 Features Are Fate: A Theory of Transfer Learning in High-Dimensional Regression Javan Tahir, Surya Ganguli, Grant M. Rotskoff

NeurIPS 2025 Rethinking Fine-Tuning When Scaling Test-Time Compute: Limiting Confidence Improves Mathematical Reasoning Feng Chen, Allan Raventos, Nan Cheng, Surya Ganguli, Shaul Druckmann

ICLRW 2025 Rethinking Fine-Tuning When Scaling Test-Time Compute: Limiting Confidence Improves Mathematical Reasoning Feng Chen, Allan Raventos, Nan Cheng, Surya Ganguli, Shaul Druckmann

NeurIPS 2024 Get Rich Quick: Exact Solutions Reveal How Unbalanced Initializations Promote Rapid Feature Learning Daniel Kunin, Allan Raventós, Clémentine Dominé, Feng Chen, David Klindt, Andrew Saxe, Surya Ganguli

ICMLW 2024 Get Rich Quick: Exact Solutions Reveal How Unbalanced Initializations Promote Rapid Feature Learning Daniel Kunin, Allan Raventos, Clémentine Carla Juliette Dominé, Feng Chen, David Klindt, Andrew M Saxe, Surya Ganguli

ICLR 2023 Disentanglement with Biological Constraints: A Theory of Functional Cell Types James C. R. Whittington, Will Dorrell, Surya Ganguli, Timothy Behrens

NeurIPSW 2023 Enhanced Cue Associated Memory in Temporally Consistent Recurrent Neural Networks Udith Haputhanthri, Liam Storan, Adam Shai, Surya Ganguli, Mark Schnitzer, Hidenori Tanaka, Fatih Dinc

TMLR 2023 Holistic Evaluation of Language Models Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michihiro Yasunaga, Yian Zhang, Deepak Narayanan, Yuhuai Wu, Ananya Kumar, Benjamin Newman, Binhang Yuan, Bobby Yan, Ce Zhang, Christian Cosgrove, Christopher D Manning, Christopher Re, Diana Acosta-Navas, Drew A. Hudson, Eric Zelikman, Esin Durmus, Faisal Ladhak, Frieda Rong, Hongyu Ren, Huaxiu Yao, Jue Wang, Keshav Santhanam, Laurel Orr, Lucia Zheng, Mert Yuksekgonul, Mirac Suzgun, Nathan Kim, Neel Guha, Niladri S. Chatterji, Omar Khattab, Peter Henderson, Qian Huang, Ryan Andrew Chi, Sang Michael Xie, Shibani Santurkar, Surya Ganguli, Tatsunori Hashimoto, Thomas Icard, Tianyi Zhang, Vishrav Chaudhary, William Wang, Xuechen Li, Yifan Mai, Yuhui Zhang, Yuta Koreeda

NeurIPS 2023 Information Geometry of the Retinal Representation Manifold Xuehao Ding, Dongsoo Lee, Joshua Melander, George Sivulka, Surya Ganguli, Stephen Baccus

NeurIPS 2023 Pretraining Task Diversity and the Emergence of Non-Bayesian In-Context Learning for Regression Allan Raventós, Mansheej Paul, Feng Chen, Surya Ganguli

ICLRW 2023 SemDeDup: Data-Efficient Learning at Web-Scale Through Semantic Deduplication Amro Kamal Mohamed Abbas, Kushal Tirumala, Daniel Simig, Surya Ganguli, Ari S. Morcos

NeurIPS 2023 Stochastic Collapse: How Gradient Noise Attracts SGD Dynamics Towards Simpler Subnetworks Feng Chen, Daniel Kunin, Atsushi Yamamura, Surya Ganguli

ICLR 2023 The Asymmetric Maximum Margin Bias of Quasi-Homogeneous Neural Networks Daniel Kunin, Atsushi Yamamura, Chao Ma, Surya Ganguli

ICLRW 2023 The Effects of Pretraining Task Diversity on In-Context Learning of Ridge Regression Allan Raventos, Mansheej Paul, Feng Chen, Surya Ganguli

ICLR 2023 Unmasking the Lottery Ticket Hypothesis: What's Encoded in a Winning Ticket's Mask? Mansheej Paul, Feng Chen, Brett W. Larsen, Jonathan Frankle, Surya Ganguli, Gintare Karolina Dziugaite

NeurIPS 2022 Beyond Neural Scaling Laws: Beating Power Law Scaling via Data Pruning Ben Sorscher, Robert Geirhos, Shashank Shekhar, Surya Ganguli, Ari Morcos

ICLR 2022 How Many Degrees of Freedom Do We Need to Train Deep Networks: A Loss Landscape Perspective Brett W Larsen, Stanislav Fort, Nic Becker, Surya Ganguli

NeurIPS 2022 Lottery Tickets on a Data Diet: Finding Initializations with Sparse Trainable Networks Mansheej Paul, Brett Larsen, Surya Ganguli, Jonathan Frankle, Gintare Karolina Dziugaite

ICLR 2022 MetaMorph: Learning Universal Controllers with Transformers Agrim Gupta, Linxi Fan, Surya Ganguli, Li Fei-Fei

ICMLW 2022 Pre-Training on a Data Diet: Identifying Sufficient Examples for Early Training Mansheej Paul, Brett W Larsen, Surya Ganguli, Jonathan Frankle, Gintare Karolina Dziugaite

NeurIPSW 2022 Unmasking the Lottery Ticket Hypothesis: Efficient Adaptive Pruning for Finding Winning Tickets Mansheej Paul, Feng Chen, Brett W. Larsen, Jonathan Frankle, Surya Ganguli, Gintare Karolina Dziugaite

ICML 2021 A Theory of High Dimensional Regression with Arbitrary Correlations Between Input Features and Target Functions: Sample Complexity, Multiple Descent Curves and a Hierarchy of Phase Transitions Gabriel Mel, Surya Ganguli

NeurIPS 2021 Deep Learning on a Data Diet: Finding Important Examples Early in Training Mansheej Paul, Surya Ganguli, Gintare Karolina Dziugaite

NeurIPS 2021 Explaining Heterogeneity in Medial Entorhinal Cortex with Task-Driven Neural Networks Aran Nayebi, Alexander Attinger, Malcolm Campbell, Kiah Hardcastle, Isabel Low, Caitlin S Mallory, Gabriel Mel, Ben Sorscher, Alex H Williams, Surya Ganguli, Lisa Giocomo, Dan Yamins

ICLR 2021 Neural Mechanics: Symmetry and Broken Conservation Laws in Deep Learning Dynamics Daniel Kunin, Javier Sagastuy-Brena, Surya Ganguli, Daniel LK Yamins, Hidenori Tanaka

ICML 2021 Understanding Self-Supervised Learning Dynamics Without Contrastive Pairs Yuandong Tian, Xinlei Chen, Surya Ganguli

NeurIPS 2020 Deep Learning Versus Kernel Learning: An Empirical Study of Loss Landscape Geometry and the Time Evolution of the Neural Tangent Kernel Stanislav Fort, Gintare Karolina Dziugaite, Mansheej Paul, Sepideh Kharaghani, Daniel M. Roy, Surya Ganguli

NeurIPS 2020 Identifying Learning Rules from Neural Network Observables Aran Nayebi, Sanjana Srivastava, Surya Ganguli, Daniel L Yamins

NeurIPS 2020 Predictive Coding in Balanced Neural Networks with Noise, Chaos and Delays Jonathan Kadmon, Jonathan Timcheck, Surya Ganguli

NeurIPS 2020 Pruning Neural Networks Without Any Data by Iteratively Conserving Synaptic Flow Hidenori Tanaka, Daniel Kunin, Daniel L Yamins, Surya Ganguli

ICML 2020 Two Routes to Scalable Credit Assignment Without Weight Symmetry Daniel Kunin, Aran Nayebi, Javier Sagastuy-Brena, Surya Ganguli, Jonathan Bloom, Daniel Yamins

NeurIPS 2019 A Unified Theory for the Origin of Grid Cells Through the Lens of Pattern Formation Ben Sorscher, Gabriel Mel, Surya Ganguli, Samuel Ocko

ICLR 2019 A Unified Theory of Early Visual Representations from Retina to Cortex Through Anatomically Constrained Deep CNNs Jack Lindsey, Samuel A. Ocko, Surya Ganguli, Stephane Deny

ICLR 2019 An Analytic Theory of Generalization Dynamics and Transfer Learning in Deep Linear Networks Andrew K. Lampinen, Surya Ganguli

NeurIPS 2019 From Deep Learning to Mechanistic Understanding in Neuroscience: The Structure of Retinal Prediction Hidenori Tanaka, Aran Nayebi, Niru Maheswaranathan, Lane McIntosh, Stephen Baccus, Surya Ganguli

ICMLW 2019 Line Attractor Dynamics in Recurrent Networks for Sentiment Classiﬁcation Niru Maheswaranathan, Alex H. Williams, Matthew D. Golub, Surya Ganguli, David Sussillo

NeurIPSW 2019 Revealing Computational Mechanisms of Retinal Prediction via Model Reduction Hidenori Tanaka, Aran Nayebi, Niru Maheswaranathan, Lane McIntosh, Stephen A. Baccus, Surya Ganguli

NeurIPS 2019 Reverse Engineering Recurrent Networks for Sentiment Classification Reveals Line Attractor Dynamics Niru Maheswaranathan, Alex Williams, Matthew Golub, Surya Ganguli, David Sussillo

NeurIPS 2019 Universality and Individuality in Neural Dynamics Across Large Populations of Recurrent Networks Niru Maheswaranathan, Alex Williams, Matthew Golub, Surya Ganguli, David Sussillo

NeurIPS 2018 Statistical Mechanics of Low-Rank Tensor Decomposition Jonathan Kadmon, Surya Ganguli

NeurIPS 2018 Task-Driven Convolutional Recurrent Models of the Visual System Aran Nayebi, Daniel Bear, Jonas Kubilius, Kohitij Kar, Surya Ganguli, David Sussillo, James J DiCarlo, Daniel L Yamins

NeurIPS 2018 The Emergence of Multiple Retinal Cell Types Through Efficient Coding of Natural Movies Samuel Ocko, Jack Lindsey, Surya Ganguli, Stephane Deny

AISTATS 2018 The Emergence of Spectral Universality in Deep Networks Jeffrey Pennington, Samuel S. Schoenholz, Surya Ganguli

ICML 2017 Continual Learning Through Synaptic Intelligence Friedemann Zenke, Ben Poole, Surya Ganguli

ICLR 2017 Deep Information Propagation Samuel S. Schoenholz, Justin Gilmer, Surya Ganguli, Jascha Sohl-Dickstein

ICLR 2017 Intelligent Synapses for Multi-Task and Transfer Learning Ben Poole, Friedemann Zenke, Surya Ganguli

ICML 2017 On the Expressive Power of Deep Neural Networks Maithra Raghu, Ben Poole, Jon Kleinberg, Surya Ganguli, Jascha Sohl-Dickstein

NeurIPS 2017 Resurrecting the Sigmoid in Deep Learning Through Dynamical Isometry: Theory and Practice Jeffrey Pennington, Samuel Schoenholz, Surya Ganguli

NeurIPS 2017 Variational Walkback: Learning a Transition Operator as a Stochastic Recurrent Net Anirudh Goyal ALIAS PARTH Goyal, Nan Rosemary Ke, Surya Ganguli, Yoshua Bengio

NeurIPS 2016 An Equivalence Between High Dimensional Bayes Optimal Inference and M-Estimation Madhu Advani, Surya Ganguli

NeurIPS 2016 Deep Learning Models of the Retinal Response to Natural Scenes Lane McIntosh, Niru Maheswaranathan, Aran Nayebi, Surya Ganguli, Stephen Baccus

NeurIPS 2016 Exponential Expressivity in Deep Neural Networks Through Transient Chaos Ben Poole, Subhaneil Lahiri, Maithra Raghu, Jascha Sohl-Dickstein, Surya Ganguli

NeurIPS 2015 Deep Knowledge Tracing Chris Piech, Jonathan Bassen, Jonathan Huang, Surya Ganguli, Mehran Sahami, Leonidas Guibas, Jascha Sohl-Dickstein

ICML 2015 Deep Unsupervised Learning Using Nonequilibrium Thermodynamics Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, Surya Ganguli

ICLR 2014 Exact Solutions to the Nonlinear Dynamics of Learning in Deep Linear Neural Networks Andrew M. Saxe, James L. McClelland, Surya Ganguli

ICML 2014 Fast Large-Scale Optimization by Unifying Stochastic Gradient and Quasi-Newton Methods Jascha Sohl-Dickstein, Ben Poole, Surya Ganguli

NeurIPS 2014 Identifying and Attacking the Saddle Point Problem in High-Dimensional Non-Convex Optimization Yann N. Dauphin, Razvan Pascanu, Caglar Gulcehre, Kyunghyun Cho, Surya Ganguli, Yoshua Bengio

NeurIPS 2013 A Memory Frontier for Complex Synapses Subhaneil Lahiri, Surya Ganguli

NeurIPS 2010 Short-Term Memory in Neuronal Networks Through Dynamical Compressed Sensing Surya Ganguli, Haim Sompolinsky