Soufiane Hayou

Publication Activity (10 Years)

Years Active: 2018-2024
Publications (10 Years): 31

Top Topics

Activation Function

Top Venues

Trans. Mach. Learn. Res.

Publications

Soufiane Hayou, Nikhil Ghosh, Bin Yu
LoRA+: Efficient Low Rank Adaptation of Large Models. CoRR (2024)
Mohamed El Amine Seddik, Suei-Wen Chen, Soufiane Hayou, Pierre Youssef, Mérouane Debbah
How Bad is Training on Synthetic Data? A Statistical Analysis of Language Model Collapse. CoRR (2024)
Greg Yang, Dingli Yu, Chen Zhu, Soufiane Hayou
Tensor Programs VI: Feature Learning in Infinite Depth Neural Networks. ICLR (2024)
Jiayuan Ye, Anastasia Borovykh, Soufiane Hayou, Reza Shokri
Leave-one-out Distinguishability in Machine Learning. ICLR (2024)
Soufiane Hayou, Nikhil Ghosh, Bin Yu
The Impact of Initialization on LoRA Finetuning Dynamics. CoRR (2024)
Greg Yang, Dingli Yu, Chen Zhu, Soufiane Hayou
Tensor Programs VI: Feature Learning in Infinite-Depth Neural Networks. CoRR (2023)
Jiayuan Ye, Anastasia Borovykh, Soufiane Hayou, Reza Shokri
Leave-one-out Distinguishability in Machine Learning. CoRR (2023)
Soufiane Hayou
On the Connection Between Riemann Hypothesis and a Special Class of Neural Networks. CoRR (2023)
Soufiane Hayou, Greg Yang
Width and Depth Limits Commute in Residual Networks. CoRR (2023)
Soufiane Hayou
Commutative Width and Depth Scaling in Deep Neural Networks. CoRR (2023)
Soufiane Hayou
On the infinite-depth limit of finite-width neural networks. Trans. Mach. Learn. Res. 2023 (2023)
Fadhel Ayed, Soufiane Hayou
Data pruning and neural scaling laws: fundamental limitations of score-based algorithms. CoRR (2023)
Soufiane Hayou, Greg Yang
Width and Depth Limits Commute in Residual Networks. ICML (2023)
Fadhel Ayed, Soufiane Hayou
Data pruning and neural scaling laws: fundamental limitations of score-based algorithms. Trans. Mach. Learn. Res. 2023 (2023)
Soufiane Hayou
On the infinite-depth limit of finite-width neural networks. CoRR (2022)
Fusheng Liu, Haizhao Yang, Soufiane Hayou, Qianxiao Li
Connecting Optimization and Generalization via Gradient Flow Path Length. CoRR (2022)
Fusheng Liu, Haizhao Yang, Soufiane Hayou, Qianxiao Li
From Optimization Dynamics to Generalization Bounds via Łojasiewicz Gradient Inequality. Trans. Mach. Learn. Res. 2022 (2022)
Yizhang Lou, Chris E. Mingard, Soufiane Hayou
Feature Learning and Signal Propagation in Deep Neural Networks. ICML (2022)
Soufiane Hayou, Jean-Francois Ton, Arnaud Doucet, Yee Whye Teh
Robust Pruning at Initialization. ICLR (2021)
Soufiane Hayou, Fadhel Ayed
Regularization in ResNet with Stochastic Depth. CoRR (2021)
Soufiane Hayou, Arnaud Doucet, Judith Rousseau
The Curse of Depth in Kernel Regime. ICBINB@NeurIPS (2021)
Soufiane Hayou, Bobby He, Gintare Karolina Dziugaite
Probabilistic fine-tuning of pruning masks and PAC-Bayes self-bounded learning. CoRR (2021)
Yizhang Lou, Chris Mingard, Soufiane Hayou
The Equilibrium Hypothesis: Rethinking implicit regularization in Deep Neural Networks. CoRR (2021)
Soufiane Hayou, Eugenio Clerico, Bobby He, George Deligiannidis, Arnaud Doucet, Judith Rousseau
Stable ResNet. AISTATS (2021)
Soufiane Hayou, Fadhel Ayed
Regularization in ResNet with Stochastic Depth. NeurIPS (2021)
Soufiane Hayou, Jean-Francois Ton, Arnaud Doucet, Yee Whye Teh
Pruning untrained neural networks: Principles and Analysis. CoRR (2020)
Soufiane Hayou, Eugenio Clerico, Bobby He, George Deligiannidis, Arnaud Doucet, Judith Rousseau
Stable ResNet. CoRR (2020)
Soufiane Hayou, Arnaud Doucet, Judith Rousseau
On the Impact of the Activation Function on Deep Neural Networks Training. CoRR (2019)
Soufiane Hayou, Arnaud Doucet, Judith Rousseau
Training Dynamics of Deep Networks using Stochastic Gradient Descent via Neural Tangent Kernel. CoRR (2019)
Soufiane Hayou, Arnaud Doucet, Judith Rousseau
On the Impact of the Activation function on Deep Neural Networks Training. ICML (2019)
Soufiane Hayou, Arnaud Doucet, Judith Rousseau
On the Selection of Initialization and Activation Function for Deep Neural Networks. CoRR (2018)