Singh, Sidak Pal

22 publications

ICML 2025 Avoiding Spurious Sharpness Minimization Broadens Applicability of SAM Sidak Pal Singh, Hossein Mobahi, Atish Agarwala, Yann Dauphin

NeurIPS 2025 Generalized Linear Mode Connectivity for Transformers Alexander Theus, Alessandro Cabodi, Sotiris Anagnostidis, Antonio Orvieto, Sidak Pal Singh, Valentina Boeva

ICLR 2025 The Directionality of Optimization Trajectories in Neural Networks Sidak Pal Singh, Bobby He, Thomas Hofmann, Bernhard Schölkopf

ICLR 2025 What Does It Mean to Be a Transformer? Insights from a Theoretical Hessian Analysis Weronika Ormaniec, Felix Dangel, Sidak Pal Singh

ICMLW 2024 Closed Form of the Hessian Spectrum for Some Neural Networks Sidak Pal Singh, Thomas Hofmann

ICMLW 2024 Hallmarks of Optimization Trajectories in Neural Networks and LLMs: Directional Exploration and Redundancy Sidak Pal Singh, Bobby He, Thomas Hofmann, Bernhard Schölkopf

ICMLW 2024 Landscaping Linear Mode Connectivity Sidak Pal Singh, Linara Adilova, Michael Kamp, Asja Fischer, Bernhard Schölkopf, Thomas Hofmann

CoLLAs 2024 Local vs Global Continual Learning Giulia Lanzillotta, Sidak Pal Singh, Benjamin F Grewe, Thomas Hofmann

AAAI 2024 Rethinking Attention: Exploring Shallow Feed-Forward Neural Networks as an Alternative to Attention Layers in Transformers (Student Abstract) Danilo Dordevic, Vukasin Bozic, Joseph Thommes, Daniele Coppola, Sidak Pal Singh

ICLR 2024 Some Fundamental Aspects About Lipschitz Continuity of Neural Networks Grigory Khromov, Sidak Pal Singh

NeurIPS 2024 Theoretical Characterisation of the Gauss Newton Conditioning in Neural Networks Jim Zhao, Sidak Pal Singh, Aurelien Lucchi

ICLR 2024 Towards Meta-Pruning via Optimal Transport Alexander Theus, Olin Geimer, Friedrich Wicke, Thomas Hofmann, Sotiris Anagnostidis, Sidak Pal Singh

ICLR 2024 Transformer Fusion with Optimal Transport Moritz Imfeld, Jacopo Graldi, Marco Giordano, Thomas Hofmann, Sotiris Anagnostidis, Sidak Pal Singh

NeurIPSW 2023 Escaping Random Teacher Initialization Enhances Signal Propagation and Representation Felix Sarnthein, Sidak Pal Singh, Antonio Orvieto, Thomas Hofmann

ICML 2023 The Hessian Perspective into the Nature of Convolutional Neural Networks Sidak Pal Singh, Thomas Hofmann, Bernhard Schölkopf

ICLR 2022 Phenomenology of Double Descent in Finite-Width Neural Networks Sidak Pal Singh, Aurelien Lucchi, Thomas Hofmann, Bernhard Schölkopf

NeurIPS 2022 Signal Propagation in Transformers: Theoretical Perspectives and the Role of Rank Collapse Lorenzo Noci, Sotiris Anagnostidis, Luca Biggio, Antonio Orvieto, Sidak Pal Singh, Aurelien Lucchi

NeurIPS 2021 Analytic Insights into Structure and Rank of Neural Network Hessian Maps Sidak Pal Singh, Gregor Bachmann, Thomas Hofmann

AISTATS 2020 Context Mover’s Distance & Barycenters: Optimal Transport of Contexts for Building Representations Sidak Pal Singh, Andreas Hug, Aymeric Dieuleveut, Martin Jaggi

NeurIPS 2020 Model Fusion via Optimal Transport Sidak Pal Singh, Martin Jaggi

NeurIPS 2020 WoodFisher: Efficient Second-Order Approximation for Neural Network Compression Sidak Pal Singh, Dan Alistarh

ICLRW 2019 Context Mover's Distance & Barycenters: Optimal Transport of Contexts for Building Representations Sidak Pal Singh, Andreas Hug, Aymeric Dieuleveut, Martin Jaggi