Surya Ganguli

ORCID

Publication Activity (10 Years)

Years Active: 2010-2024
Publications (10 Years): 102

Top Venues

Publications

Aditya Cowsik, Tamra Nebabu, Xiao-Liang Qi, Surya Ganguli
Geometric Dynamics of Signal Propagation Predict Trainability of Transformers. CoRR (2024)
Daniel Kunin, Allan Raventós, Clémentine Dominé, Feng Chen, David Klindt, Andrew M. Saxe, Surya Ganguli
Get rich quick: exact solutions reveal how unbalanced initializations promote rapid feature learning. CoRR (2024)
Daniel Kunin, Javier Sagastuy-Breña, Lauren Gillespie, Eshed Margalit, Hidenori Tanaka, Surya Ganguli, Daniel L. K. Yamins
The Limiting Dynamics of SGD: Modified Loss, Phase-Space Oscillations, and Anomalous Diffusion. Neural Comput. 36 (1) (2024)
James C. R. Whittington, Will Dorrell, Surya Ganguli, Timothy Behrens
Disentanglement with Biological Constraints: A Theory of Functional Cell Types. ICLR (2023)
Allan Raventós, Mansheej Paul, Feng Chen, Surya Ganguli
Pretraining task diversity and the emergence of non-Bayesian in-context learning for regression. CoRR (2023)
Mansheej Paul, Feng Chen, Brett W. Larsen, Jonathan Frankle, Surya Ganguli, Gintare Karolina Dziugaite
Unmasking the Lottery Ticket Hypothesis: What's Encoded in a Winning Ticket's Mask? ICLR (2023)
Xuehao Ding, Dongsoo Lee, Joshua Melander, George Sivulka, Surya Ganguli, Stephen Baccus
Information Geometry of the Retinal Representation Manifold. NeurIPS (2023)
Allan Raventós, Mansheej Paul, Feng Chen, Surya Ganguli
Pretraining task diversity and the emergence of non-Bayesian in-context learning for regression. NeurIPS (2023)
Feng Chen, Daniel Kunin, Atsushi Yamamura, Surya Ganguli
Stochastic Collapse: How Gradient Noise Attracts SGD Dynamics Towards Simpler Subnetworks. NeurIPS (2023)
Daniel Kunin, Atsushi Yamamura, Chao Ma, Surya Ganguli
The Asymmetric Maximum Margin Bias of Quasi-Homogeneous Neural Networks. ICLR (2023)
Feng Chen, Daniel Kunin, Atsushi Yamamura, Surya Ganguli
Stochastic Collapse: How Gradient Noise Attracts SGD Dynamics Towards Simpler Subnetworks. CoRR (2023)
Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michihiro Yasunaga, Yian Zhang, Deepak Narayanan, Yuhuai Wu, Ananya Kumar, Benjamin Newman, Binhang Yuan, Bobby Yan, Ce Zhang, Christian Cosgrove, Christopher D. Manning, Christopher Ré, Diana Acosta-Navas, Drew A. Hudson, Eric Zelikman, Esin Durmus, Faisal Ladhak, Frieda Rong, Hongyu Ren, Huaxiu Yao, Jue Wang, Keshav Santhanam, Laurel J. Orr, Lucia Zheng, Mert Yüksekgönül, Mirac Suzgun, Nathan Kim, Neel Guha, Niladri S. Chatterji, Omar Khattab, Peter Henderson, Qian Huang, Ryan Chi, Sang Michael Xie, Shibani Santurkar, Surya Ganguli, Tatsunori Hashimoto, Thomas Icard, Tianyi Zhang, Vishrav Chaudhary, William Wang, Xuechen Li, Yifan Mai, Yuhui Zhang, Yuta Koreeda
Holistic Evaluation of Language Models. Trans. Mach. Learn. Res. 2023 (2023)
Amro Abbas, Kushal Tirumala, Daniel Simig, Surya Ganguli, Ari S. Morcos
SemDeDup: Data-efficient learning at web-scale through semantic deduplication. CoRR (2023)
James C. R. Whittington, Will Dorrell, Surya Ganguli, Timothy Edward John Behrens
Disentangling with Biological Constraints: A Theory of Functional Cell Types. CoRR (2022)
Brett W. Larsen, Stanislav Fort, Nic Becker, Surya Ganguli
How many degrees of freedom do we need to train deep networks: a loss landscape perspective. ICLR (2022)
Ben Sorscher, Robert Geirhos, Shashank Shekhar, Surya Ganguli, Ari Morcos
Beyond neural scaling laws: beating power law scaling via data pruning. NeurIPS (2022)
Aran Nayebi, Javier Sagastuy-Breña, Daniel M. Bear, Kohitij Kar, Jonas Kubilius, Surya Ganguli, David Sussillo, James J. DiCarlo, Daniel L. K. Yamins
Recurrent Connections in the Primate Ventral Visual Stream Mediate a Trade-Off Between Task Performance and Network Size During Core Object Recognition. Neural Comput. 34 (8) (2022)
Mansheej Paul, Feng Chen, Brett W. Larsen, Jonathan Frankle, Surya Ganguli, Gintare Karolina Dziugaite
Unmasking the Lottery Ticket Hypothesis: What's Encoded in a Winning Ticket's Mask? CoRR (2022)
Agrim Gupta, Linxi Fan, Surya Ganguli, Li Fei-Fei
MetaMorph: Learning Universal Controllers with Transformers. ICLR (2022)
Ben Sorscher, Robert Geirhos, Shashank Shekhar, Surya Ganguli, Ari S. Morcos
Beyond neural scaling laws: beating power law scaling via data pruning. CoRR (2022)
Daniel Kunin, Atsushi Yamamura, Chao Ma, Surya Ganguli
The Asymmetric Maximum Margin Bias of Quasi-Homogeneous Neural Networks. CoRR (2022)
Stanislav Fort, Ekin Dogus Cubuk, Surya Ganguli, Samuel S. Schoenholz
What does a deep neural network confidently perceive? The effective dimension of high certainty class manifolds and their low confidence boundaries. CoRR (2022)
Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michihiro Yasunaga, Yian Zhang, Deepak Narayanan, Yuhuai Wu, Ananya Kumar, Benjamin Newman, Binhang Yuan, Bobby Yan, Ce Zhang, Christian Cosgrove, Christopher D. Manning, Christopher Ré, Diana Acosta-Navas, Drew A. Hudson, Eric Zelikman, Esin Durmus, Faisal Ladhak, Frieda Rong, Hongyu Ren, Huaxiu Yao, Jue Wang, Keshav Santhanam, Laurel J. Orr, Lucia Zheng, Mert Yüksekgönül, Mirac Suzgun, Nathan Kim, Neel Guha, Niladri S. Chatterji, Omar Khattab, Peter Henderson, Qian Huang, Ryan Chi, Sang Michael Xie, Shibani Santurkar, Surya Ganguli, Tatsunori Hashimoto, Thomas Icard, Tianyi Zhang, Vishrav Chaudhary, William Wang, Xuechen Li, Yifan Mai, Yuhui Zhang, Yuta Koreeda
Holistic Evaluation of Language Models. CoRR (2022)
Anthony Zador, Blake A. Richards, Bence Ölveczky, Sean Escola, Yoshua Bengio, Kwabena Boahen, Matthew M. Botvinick, Dmitri B. Chklovskii, Anne Churchland, Claudia Clopath, James DiCarlo, Surya Ganguli, Jeff Hawkins, Konrad P. Körding, Alexei A. Koulakov, Yann LeCun, Timothy P. Lillicrap, Adam H. Marblestone, Bruno A. Olshausen, Alexandre Pouget, Cristina Savin, Terrence J. Sejnowski, Eero P. Simoncelli, Sara A. Solla, David Sussillo, Andreas S. Tolias, Doris Tsao
Toward Next-Generation Artificial Intelligence: Catalyzing the NeuroAI Revolution. CoRR (2022)
Agrim Gupta, Linxi Fan, Surya Ganguli, Li Fei-Fei
MetaMorph: Learning Universal Controllers with Transformers. CoRR (2022)
Mansheej Paul, Brett W. Larsen, Surya Ganguli, Jonathan Frankle, Gintare Karolina Dziugaite
Lottery Tickets on a Data Diet: Finding Initializations with Sparse Trainable Networks. CoRR (2022)
Christopher H. Stock, Sarah E. Harvey, Samuel A. Ocko, Surya Ganguli
Synaptic balancing: A biologically plausible local learning rule that provably increases neural network noise robustness without sacrificing task performance. PLoS Comput. Biol. 18 (9) (2022)
Jonathan Timcheck, Jonathan Kadmon, Kwabena Boahen, Surya Ganguli
Optimal noise level for coding with tightly balanced networks of spiking neurons in the presence of transmission delays. PLoS Comput. Biol. 18 (10) (2022)
Mansheej Paul, Brett W. Larsen, Surya Ganguli, Jonathan Frankle, Gintare Karolina Dziugaite
Lottery Tickets on a Data Diet: Finding Initializations with Sparse Trainable Networks. NeurIPS (2022)
Aran Nayebi, Alexander Attinger, Malcolm Campbell, Kiah Hardcastle, Isabel Low, Caitlin S. Mallory, Gabriel Mel, Ben Sorscher, Alex H. Williams, Surya Ganguli, Lisa M. Giocomo, Daniel L. K. Yamins
Explaining heterogeneity in medial entorhinal cortex with task-driven neural networks. NeurIPS (2021)
Daniel Kunin, Javier Sagastuy-Breña, Surya Ganguli, Daniel L. Yamins, Hidenori Tanaka
Neural Mechanics: Symmetry and Broken Conservation Laws in Deep Learning Dynamics. ICLR (2021)
Mansheej Paul, Surya Ganguli, Gintare Karolina Dziugaite
Deep Learning on a Data Diet: Finding Important Examples Early in Training. NeurIPS (2021)
Daniel Kunin, Javier Sagastuy-Breña, Lauren Gillespie, Eshed Margalit, Hidenori Tanaka, Surya Ganguli, Daniel L. K. Yamins
Rethinking the limiting dynamics of SGD: modified loss, phase space oscillations, and anomalous diffusion. CoRR (2021)
Yuandong Tian, Xinlei Chen, Surya Ganguli
Understanding self-supervised Learning Dynamics without Contrastive Pairs. CoRR (2021)
Mansheej Paul, Surya Ganguli, Gintare Karolina Dziugaite
Deep Learning on a Data Diet: Finding Important Examples Early in Training. CoRR (2021)
Gabriel Mel, Surya Ganguli
A theory of high dimensional regression with arbitrary correlations between input features and target functions: sample complexity, multiple descent curves and a hierarchy of phase transitions. ICML (2021)
Brett W. Larsen, Stanislav Fort, Nic Becker, Surya Ganguli
How many degrees of freedom do we need to train deep networks: a loss landscape perspective. CoRR (2021)
Yuandong Tian, Xinlei Chen, Surya Ganguli
Understanding self-supervised learning dynamics without contrastive pairs. ICML (2021)
Agrim Gupta, Silvio Savarese, Surya Ganguli, Li Fei-Fei
Embodied Intelligence via Learning and Evolution. CoRR (2021)
Daniel Kunin, Aran Nayebi, Javier Sagastuy-Breña, Surya Ganguli, Jonathan M. Bloom, Daniel L. K. Yamins
Two Routes to Scalable Credit Assignment without Weight Symmetry. CoRR (2020)
Jonathan Kadmon, Jonathan Timcheck, Surya Ganguli
Predictive coding in balanced neural networks with noise, chaos and delays. NeurIPS (2020)
Stanislav Fort, Gintare Karolina Dziugaite, Mansheej Paul, Sepideh Kharaghani, Daniel M. Roy, Surya Ganguli
Deep learning versus kernel learning: an empirical study of loss landscape geometry and the time evolution of the Neural Tangent Kernel. NeurIPS (2020)
Stanislav Fort, Gintare Karolina Dziugaite, Mansheej Paul, Sepideh Kharaghani, Daniel M. Roy, Surya Ganguli
Deep learning versus kernel learning: an empirical study of loss landscape geometry and the time evolution of the Neural Tangent Kernel. CoRR (2020)
Daniel Kunin, Javier Sagastuy-Breña, Surya Ganguli, Daniel L. K. Yamins, Hidenori Tanaka
Neural Mechanics: Symmetry and Broken Conservation Laws in Deep Learning Dynamics. CoRR (2020)
John Hewitt, Michael Hahn, Surya Ganguli, Percy Liang, Christopher D. Manning
RNNs can generate bounded hierarchical languages with optimal memory. CoRR (2020)
John Hewitt, Michael Hahn, Surya Ganguli, Percy Liang, Christopher D. Manning
RNNs can generate bounded hierarchical languages with optimal memory. EMNLP (1) (2020)
Yuandong Tian, Lantao Yu, Xinlei Chen, Surya Ganguli
Understanding Self-supervised Learning with Dual Deep Networks. CoRR (2020)
Aran Nayebi, Sanjana Srivastava, Surya Ganguli, Daniel L. Yamins
Identifying Learning Rules From Neural Network Observables. NeurIPS (2020)
Hidenori Tanaka, Daniel Kunin, Daniel L. Yamins, Surya Ganguli
Pruning neural networks without any data by iteratively conserving synaptic flow. NeurIPS (2020)
Daniel Kunin, Aran Nayebi, Javier Sagastuy-Breña, Surya Ganguli, Jonathan M. Bloom, Daniel Yamins
Two Routes to Scalable Credit Assignment without Weight Symmetry. ICML (2020)
Hidenori Tanaka, Daniel Kunin, Daniel L. K. Yamins, Surya Ganguli
Pruning neural networks without any data by iteratively conserving synaptic flow. CoRR (2020)
Oleg I. Rumyantsev, Jérôme A. Lecoq, Oscar Hernandez, Yanping Zhang, Joan Savall, Radoslaw Chrapkiewicz, Jane Li, Hongkui Zeng, Surya Ganguli, Mark J. Schnitzer
Fundamental bounds on the fidelity of sensory cortical coding. Nat. 580 (7801) (2020)
Aran Nayebi, Sanjana Srivastava, Surya Ganguli, Daniel L. K. Yamins
Identifying Learning Rules From Neural Network Observables. CoRR (2020)
Andrew K. Lampinen, Surya Ganguli
An analytic theory of generalization dynamics and transfer learning in deep linear networks. ICLR (Poster) (2019)
Stanislav Fort, Surya Ganguli
Emergent properties of the local geometry of neural loss landscapes. CoRR (2019)
Jack Lindsey, Samuel A. Ocko, Surya Ganguli, Stéphane Deny
A Unified Theory of Early Visual Representations from Retina to Cortex through Anatomically Constrained Deep CNNs. ICLR (2019)
Anthony Degleris, Ben Antin, Surya Ganguli, Alex H. Williams
Fast Convolutive Nonnegative Matrix Factorization Through Coordinate and Block Coordinate Updates. CoRR (2019)
Ben Sorscher, Gabriel Mel, Surya Ganguli, Samuel A. Ocko
A unified theory for the origin of grid cells through the lens of pattern formation. NeurIPS (2019)
Niru Maheswaranathan, Alex H. Williams, Matthew D. Golub, Surya Ganguli, David Sussillo
Universality and individuality in neural dynamics across large populations of recurrent networks. NeurIPS (2019)
Hidenori Tanaka, Aran Nayebi, Niru Maheswaranathan, Lane McIntosh, Stephen A. Baccus, Surya Ganguli
From deep learning to mechanistic understanding in neuroscience: the structure of retinal prediction. CoRR (2019)
Jack Lindsey, Samuel A. Ocko, Surya Ganguli, Stéphane Deny
A Unified Theory of Early Visual Representations from Retina to Cortex through Anatomically Constrained Deep CNNs. CoRR (2019)
Hidenori Tanaka, Aran Nayebi, Niru Maheswaranathan, Lane McIntosh, Stephen Baccus, Surya Ganguli
From deep learning to mechanistic understanding in neuroscience: the structure of retinal prediction. NeurIPS (2019)
Niru Maheswaranathan, Alex H. Williams, Matthew D. Golub, Surya Ganguli, David Sussillo
Reverse engineering recurrent networks for sentiment classification reveals line attractor dynamics. CoRR (2019)
Niru Maheswaranathan, Alex H. Williams, Matthew D. Golub, Surya Ganguli, David Sussillo
Universality and individuality in neural dynamics across large populations of recurrent networks. CoRR (2019)
Niru Maheswaranathan, Alex H. Williams, Matthew D. Golub, Surya Ganguli, David Sussillo
Reverse engineering recurrent networks for sentiment classification reveals line attractor dynamics. NeurIPS (2019)
Jonathan Kadmon, Surya Ganguli
Statistical mechanics of low-rank tensor decomposition. CoRR (2018)
Jonathan Kadmon, Surya Ganguli
Statistical mechanics of low-rank tensor decomposition. NeurIPS (2018)
Andrew M. Saxe, James L. McClelland, Surya Ganguli
A mathematical theory of semantic development in deep neural networks. CoRR (2018)
Jeffrey Pennington, Samuel S. Schoenholz, Surya Ganguli
The emergence of spectral universality in deep networks. AISTATS (2018)
Niru Maheswaranathan, David B. Kastner, Stephen A. Baccus, Surya Ganguli
Inferring hidden structure in multilayered neural circuits. PLoS Comput. Biol. 14 (8) (2018)
Andrew K. Lampinen, Surya Ganguli
An analytic theory of generalization dynamics and transfer learning in deep linear networks. CoRR (2018)
Jeffrey Pennington, Samuel S. Schoenholz, Surya Ganguli
The Emergence of Spectral Universality in Deep Networks. CoRR (2018)
Samuel A. Ocko, Jack Lindsey, Surya Ganguli, Stéphane Deny
The emergence of multiple retinal cell types through efficient coding of natural movies. NeurIPS (2018)
Aran Nayebi, Daniel Bear, Jonas Kubilius, Kohitij Kar, Surya Ganguli, David Sussillo, James J. DiCarlo, Daniel L. K. Yamins
Task-Driven Convolutional Recurrent Models of the Visual System. CoRR (2018)
Friedemann Zenke, Surya Ganguli
SuperSpike: Supervised Learning in Multilayer Spiking Neural Networks. Neural Comput. 30 (6) (2018)
Aran Nayebi, Daniel Bear, Jonas Kubilius, Kohitij Kar, Surya Ganguli, David Sussillo, James J. DiCarlo, Daniel L. Yamins
Task-Driven Convolutional Recurrent Models of the Visual System. NeurIPS (2018)
Samuel S. Schoenholz, Justin Gilmer, Surya Ganguli, Jascha Sohl-Dickstein
Deep Information Propagation. ICLR (Poster) (2017)
Jeffrey Pennington, Samuel S. Schoenholz, Surya Ganguli
Resurrecting the sigmoid in deep learning through dynamical isometry: theory and practice. CoRR (2017)
Jeffrey Pennington, Samuel S. Schoenholz, Surya Ganguli
Resurrecting the sigmoid in deep learning through dynamical isometry: theory and practice. NIPS (2017)
Ben Poole, Friedemann Zenke, Surya Ganguli
Intelligent synapses for multi-task and transfer learning. ICLR (Workshop) (2017)
Aran Nayebi, Surya Ganguli
Biologically inspired protection of deep networks from adversarial attacks. CoRR (2017)
Maithra Raghu, Ben Poole, Jon M. Kleinberg, Surya Ganguli, Jascha Sohl-Dickstein
On the Expressive Power of Deep Neural Networks. ICML (2017)
Friedemann Zenke, Ben Poole, Surya Ganguli
Improved multitask learning through synaptic intelligence. CoRR (2017)
Anirudh Goyal, Nan Rosemary Ke, Surya Ganguli, Yoshua Bengio
Variational Walkback: Learning a Transition Operator as a Stochastic Recurrent Net. NIPS (2017)
Anirudh Goyal, Nan Rosemary Ke, Surya Ganguli, Yoshua Bengio
Variational Walkback: Learning a Transition Operator as a Stochastic Recurrent Net. CoRR (2017)
Benjamin Naecker, Niru Maheswaranathan, Surya Ganguli, Stephen Baccus
Pyret: A Python package for analysis of neurophysiology data. J. Open Source Softw. 2 (9) (2017)
Friedemann Zenke, Surya Ganguli
SuperSpike: Supervised learning in multi-layer spiking neural networks. CoRR (2017)
Friedemann Zenke, Ben Poole, Surya Ganguli
Continual Learning Through Synaptic Intelligence. ICML (2017)
Ben Poole, Subhaneil Lahiri, Maithra Raghu, Jascha Sohl-Dickstein, Surya Ganguli
Exponential expressivity in deep neural networks through transient chaos. CoRR (2016)
Ben Poole, Subhaneil Lahiri, Maithra Raghu, Jascha Sohl-Dickstein, Surya Ganguli
Exponential expressivity in deep neural networks through transient chaos. NIPS (2016)
Maithra Raghu, Ben Poole, Jon M. Kleinberg, Surya Ganguli, Jascha Sohl-Dickstein
On the expressive power of deep neural networks. CoRR (2016)
Subhaneil Lahiri, Jascha Sohl-Dickstein, Surya Ganguli
A universal tradeoff between power, precision and speed in physical communication. CoRR (2016)
Samuel S. Schoenholz, Justin Gilmer, Surya Ganguli, Jascha Sohl-Dickstein
Deep Information Propagation. CoRR (2016)
Lane McIntosh, Niru Maheswaranathan, Aran Nayebi, Surya Ganguli, Stephen Baccus
Deep Learning Models of the Retinal Response to Natural Scenes. NIPS (2016)
Maithra Raghu, Ben Poole, Jon M. Kleinberg, Surya Ganguli, Jascha Sohl-Dickstein
Survey of Expressivity in Deep Neural Networks. CoRR (2016)
Madhu Advani, Surya Ganguli
An equivalence between high dimensional Bayes optimal inference and M-estimation. NIPS (2016)
Subhaneil Lahiri, Peiran Gao, Surya Ganguli
Random projections of random manifolds. CoRR (2016)
Jascha Sohl-Dickstein, Eric A. Weiss, Niru Maheswaranathan, Surya Ganguli
Deep Unsupervised Learning using Nonequilibrium Thermodynamics. CoRR (2015)
Jascha Sohl-Dickstein, Eric A. Weiss, Niru Maheswaranathan, Surya Ganguli
Deep Unsupervised Learning using Nonequilibrium Thermodynamics. ICML (2015)
Chris Piech, Jonathan Bassen, Jonathan Huang, Surya Ganguli, Mehran Sahami, Leonidas J. Guibas, Jascha Sohl-Dickstein
Deep Knowledge Tracing. NIPS (2015)

Surya Ganguli.css-eqlu8i{font-size:small;padding-left:4px;}

Publication Activity (10 Years)

Top Topics

Top Venues

Surya Ganguli