Dangel, Felix

17 publications

NeurIPS 2025 Collapsing Taylor Mode Automatic Differentiation Felix Dangel, Tim Siebert, Marius Zeinhofer, Andrea Walther

ICML 2025 Fishers for Free? Approximating the Fisher Information Matrix by Recycling the Squared Gradient Accumulator Yu Xin Li, Felix Dangel, Derek Tam, Colin Raffel

ICML 2025 Hide & Seek: Transformer Symmetries Obscure Sharpness & Riemannian Geometry Finds It Marvin F. Da Silva, Felix Dangel, Sageev Oore

NeurIPS 2025 Improving Energy Natural Gradient Descent Through Woodbury, Momentum, and Randomization Andres Guzman-Cordero, Felix Dangel, Gil Goldshlager, Marius Zeinhofer

ICLR 2025 What Does It Mean to Be a Transformer? Insights from a Theoretical Hessian Analysis Weronika Ormaniec, Felix Dangel, Sidak Pal Singh

ICML 2024 Can We Remove the Square-Root in Adaptive Gradient Methods? a Second-Order Perspective Wu Lin, Felix Dangel, Runa Eschenhagen, Juhan Bae, Richard E. Turner, Alireza Makhzani

NeurIPS 2024 Convolutions and More as Einsum: A Tensor Network Perspective with Advances for Second-Order Methods Felix Dangel

NeurIPS 2024 Kronecker-Factored Approximate Curvature for Physics-Informed Neural Networks Felix Dangel, Johannes Müller, Marius Zeinhofer

ICMLW 2024 Lowering PyTorch's Memory Consumption for Selective Differentiation Samarth Bhatia, Felix Dangel

ICML 2024 Revisiting Scalable Hessian Diagonal Approximations for Applications in Reinforcement Learning Mohamed Elsayed, Homayoon Farrahi, Felix Dangel, A. Rupam Mahmood

ICML 2024 Structured Inverse-Free Natural Gradient Descent: Memory-Efficient & Numerically-Stable KFAC Wu Lin, Felix Dangel, Runa Eschenhagen, Kirill Neklyudov, Agustinus Kristiadi, Richard E. Turner, Alireza Makhzani

NeurIPSW 2023 Structured Inverse-Free Natural Gradient: Memory-Efficient & Numerically-Stable KFAC for Large Neural Nets Wu Lin, Felix Dangel, Runa Eschenhagen, Kirill Neklyudov, Agustinus Kristiadi, Richard E. Turner, Alireza Makhzani

NeurIPS 2023 The Geometry of Neural Nets' Parameter Spaces Under Reparametrization Agustinus Kristiadi, Felix Dangel, Philipp Hennig

TMLR 2023 ViViT: Curvature Access Through the Generalized Gauss-Newton’s Low-Rank Structure Felix Dangel, Lukas Tatzel, Philipp Hennig

NeurIPS 2021 Cockpit: A Practical Debugging Tool for the Training of Deep Neural Networks Frank Schneider, Felix Dangel, Philipp Hennig

AISTATS 2020 Modular Block-Diagonal Curvature Approximations for Feedforward Architectures Felix Dangel, Stefan Harmeling, Philipp Hennig

ICLR 2020 BackPACK: Packing More into Backprop Felix Dangel, Frederik Kunstner, Philipp Hennig