Hayou, Soufiane

19 publications

ICLR 2025 Maximizing the Potential of Synthetic Data: Insights from Random Matrix Theory Aymane El Firdoussi, Mohamed El Amine Seddik, Soufiane Hayou, Reda Alami, Ahmed Alzubaidi, Hakim Hacid

ICLRW 2025 Synthetic Data Pruning in High Dimensions: A Random Matrix Perspective Aymane El Firdoussi, Mohamed El Amine Seddik, Soufiane Hayou, Reda Alami, Ahmed Alzubaidi, Hakim Hacid

JMLR 2024 Commutative Scaling of Width and Depth in Deep Neural Networks Soufiane Hayou

ICLR 2024 Leave-One-Out Distinguishability in Machine Learning Jiayuan Ye, Anastasia Borovykh, Soufiane Hayou, Reza Shokri

ICML 2024 LoRA+: Efficient Low Rank Adaptation of Large Models Soufiane Hayou, Nikhil Ghosh, Bin Yu

ICLR 2024 Tensor Programs VI: Feature Learning in Infinite Depth Neural Networks Greg Yang, Dingli Yu, Chen Zhu, Soufiane Hayou

NeurIPS 2024 The Impact of Initialization on LoRA Finetuning Dynamics Soufiane Hayou, Nikhil Ghosh, Bin Yu

TMLR 2023 Data Pruning and Neural Scaling Laws: Fundamental Limitations of Score-Based Algorithms Fadhel Ayed, Soufiane Hayou

NeurIPSW 2023 Feature Learning in Infinite-Depth Neural Networks Greg Yang, Dingli Yu, Chen Zhu, Soufiane Hayou

TMLR 2023 On the Infinite-Depth Limit of Finite-Width Neural Networks Soufiane Hayou

ICML 2023 Width and Depth Limits Commute in Residual Networks Soufiane Hayou, Greg Yang

ICML 2022 Feature Learning and Signal Propagation in Deep Neural Networks Yizhang Lou, Chris E Mingard, Soufiane Hayou

TMLR 2022 From Optimization Dynamics to Generalization Bounds via Łojasiewicz Gradient Inequality Fusheng Liu, Haizhao Yang, Soufiane Hayou, Qianxiao Li

NeurIPSW 2022 The Curse of (non)convexity: The Case of an Optimization-Inspired Data Pruning Algorithm Fadhel Ayed, Soufiane Hayou

AISTATS 2021 Stable ResNet Soufiane Hayou, Eugenio Clerico, Bobby He, George Deligiannidis, Arnaud Doucet, Judith Rousseau

NeurIPS 2021 Regularization in ResNet with Stochastic Depth Soufiane Hayou, Fadhel Ayed

ICLR 2021 Robust Pruning at Initialization Soufiane Hayou, Jean-Francois Ton, Arnaud Doucet, Yee Whye Teh

NeurIPSW 2021 The Curse of Depth in Kernel Regime Soufiane Hayou, Arnaud Doucet, Judith Rousseau

ICML 2019 On the Impact of the Activation Function on Deep Neural Networks Training Soufiane Hayou, Arnaud Doucet, Judith Rousseau