Jascha Sohl-Dickstein

Publication Activity (10 Years)

Years Active: 2009-2024
Publications (10 Years): 125

Top Venues

CoRR

ICML

ICLR (Poster)

NeurIPS

University of California, Berkeley, USA (PHD)

Education History

University of California, Berkeley, USA

PHD

Publications

Jascha Sohl-Dickstein
The boundary of neural network trainability is fractal. CoRR (2024)
Mitchell Wortsman, Peter J. Liu, Lechao Xiao, Katie E. Everett, Alexander A. Alemi, Ben Adlam, John D. Co-Reyes, Izzeddin Gur, Abhishek Kumar, Roman Novak, Jeffrey Pennington, Jascha Sohl-Dickstein, Kelvin Xu, Jaehoon Lee, Justin Gilmer, Simon Kornblith
Small-scale proxies for large-scale Transformer training instabilities. ICLR (2024)
Avi Singh, John D. Co-Reyes, Rishabh Agarwal, Ankesh Anand, Piyush Patil, Xavier Garcia, Peter J. Liu, James Harrison, Jaehoon Lee, Kelvin Xu, Aaron T. Parisi, Abhishek Kumar, Alexander A. Alemi, Alex Rizkowsky, Azade Nova, Ben Adlam, Bernd Bohnet, Gamaleldin Fathy Elsayed, Hanie Sedghi, Igor Mordatch, Isabelle Simpson, Izzeddin Gur, Jasper Snoek, Jeffrey Pennington, Jiri Hron, Kathleen Kenealy, Kevin Swersky, Kshiteej Mahajan, Laura Culp, Lechao Xiao, Maxwell L. Bileschi, Noah Constant, Roman Novak, Rosanne Liu, Tris Warkentin, Yundi Qian, Yamini Bansal, Ethan Dyer, Behnam Neyshabur, Jascha Sohl-Dickstein, Noah Fiedel
Beyond Human Data: Scaling Self-Training for Problem-Solving with Language Models. Trans. Mach. Learn. Res. 2024 (2024)
Brian Lester, Jaehoon Lee, Alex Alemi, Jeffrey Pennington, Adam Roberts, Jascha Sohl-Dickstein, Noah Constant
Training LLMs over Neurally Compressed Text. CoRR (2024)
Katie Everett, Lechao Xiao, Mitchell Wortsman, Alexander A. Alemi, Roman Novak, Peter J. Liu, Izzeddin Gur, Jascha Sohl-Dickstein, Leslie Pack Kaelbling, Jaehoon Lee, Jeffrey Pennington
Scaling Exponents Across Parameterizations and Optimizers. CoRR (2024)
Avi Singh, John D. Co-Reyes, Rishabh Agarwal, Ankesh Anand, Piyush Patil, Xavier Garcia, Peter J. Liu, James Harrison, Jaehoon Lee, Kelvin Xu, Aaron Parisi, Abhishek Kumar, Alex Alemi, Alex Rizkowsky, Azade Nova, Ben Adlam, Bernd Bohnet, Gamaleldin F. Elsayed, Hanie Sedghi, Igor Mordatch, Isabelle Simpson, Izzeddin Gur, Jasper Snoek, Jeffrey Pennington, Jiri Hron, Kathleen Kenealy, Kevin Swersky, Kshiteej Mahajan, Laura Culp, Lechao Xiao, Maxwell L. Bileschi, Noah Constant, Roman Novak, Rosanne Liu, Tris Warkentin, Yundi Qian, Yamini Bansal, Ethan Dyer, Behnam Neyshabur, Jascha Sohl-Dickstein, Noah Fiedel
Beyond Human Data: Scaling Self-Training for Problem-Solving with Language Models. CoRR (2023)
Yilun Du, Conor Durkan, Robin Strudel, Joshua B. Tenenbaum, Sander Dieleman, Rob Fergus, Jascha Sohl-Dickstein, Arnaud Doucet, Will Grathwohl
Reduce, Reuse, Recycle: Compositional Generation with Energy-Based Diffusion Models and MCMC. CoRR (2023)
Oscar Li, James Harrison, Jascha Sohl-Dickstein, Virginia Smith, Luke Metz
Noise-Reuse in Online Evolution Strategies. CoRR (2023)
Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam R. Brown, Adam Santoro, Aditya Gupta, Adrià Garriga-Alonso, Agnieszka Kluska, Aitor Lewkowycz, Akshat Agarwal, Alethea Power, Alex Ray, Alex Warstadt, Alexander W. Kocurek, Ali Safaya, Ali Tazarv, Alice Xiang, Alicia Parrish, Allen Nie, Aman Hussain, Amanda Askell, Amanda Dsouza, Ambrose Slone, Ameet Rahane, Anantharaman S. Iyer, Anders Andreassen, Andrea Madotto, Andrea Santilli, Andreas Stuhlmüller, Andrew M. Dai, Andrew La, Andrew K. Lampinen, Andy Zou, Angela Jiang, Angelica Chen, Anh Vuong, Animesh Gupta, Anna Gottardi, Antonio Norelli, Anu Venkatesh, Arash Gholamidavoodi, Arfa Tabassum, Arul Menezes, Arun Kirubarajan, Asher Mullokandov, Ashish Sabharwal, Austin Herrick, Avia Efrat, Aykut Erdem, Ayla Karakas, B. Ryan Roberts, Bao Sheng Loe, Barret Zoph, Bartlomiej Bojanowski, Batuhan Özyurt, Behnam Hedayatnia, Behnam Neyshabur, Benjamin Inden, Benno Stein, Berk Ekmekci, Bill Yuchen Lin, Blake Howald, Bryan Orinion, Cameron Diao, Cameron Dour, Catherine Stinson, Cedrick Argueta, Cèsar Ferri Ramírez, Chandan Singh, Charles Rathkopf, Chenlin Meng, Chitta Baral, Chiyu Wu, Chris Callison-Burch, Chris Waites, Christian Voigt, Christopher D. Manning, Christopher Potts, Cindy Ramirez, Clara E. Rivera, Clemencia Siro, Colin Raffel, Courtney Ashcraft, Cristina Garbacea, Damien Sileo, Dan Garrette, Dan Hendrycks, Dan Kilman, Dan Roth, Daniel Freeman, Daniel Khashabi, Daniel Levy, Daniel Moseguí González, Danielle Perszyk, Danny Hernandez, Danqi Chen, Daphne Ippolito, Dar Gilboa, David Dohan, David Drakard, David Jurgens, Debajyoti Datta, Deep Ganguli, Denis Emelin, Denis Kleyko, Deniz Yuret, Derek Chen, Derek Tam, Dieuwke Hupkes, Diganta Misra, Dilyar Buzan, Dimitri Coelho Mollo, Diyi Yang, Dong-Ho Lee, Dylan Schrader, Ekaterina Shutova, Ekin Dogus Cubuk, Elad Segal, Eleanor Hagerman, Elizabeth Barnes, Elizabeth Donoway, Ellie Pavlick, Emanuele Rodolà, Emma Lam, Eric Chu, Eric Tang, Erkut Erdem, Ernie Chang, Ethan A. Chi, Ethan Dyer, Ethan J. Jerzak, Ethan Kim, Eunice Engefu Manyasi, Evgenii Zheltonozhskii, Fanyue Xia, Fatemeh Siar, Fernando Martínez-Plumed, Francesca Happé, François Chollet, Frieda Rong, Gaurav Mishra, Genta Indra Winata, Gerard de Melo, Germán Kruszewski, Giambattista Parascandolo, Giorgio Mariani, Gloria Wang, Gonzalo Jaimovitch-López, Gregor Betz, Guy Gur-Ari, Hana Galijasevic, Hannah Kim, Hannah Rashkin, Hannaneh Hajishirzi, Harsh Mehta, Hayden Bogar, Henry Shevlin, Hinrich Schütze, Hiromu Yakura, Hongming Zhang, Hugh Mee Wong, Ian Ng, Isaac Noble, Jaap Jumelet, Jack Geissinger, Jackson Kernion, Jacob Hilton, Jaehoon Lee, Jaime Fernández Fisac, James B. Simon, James Koppel, James Zheng, James Zou, Jan Kocon, Jana Thompson, Janelle Wingfield, Jared Kaplan, Jarema Radom, Jascha Sohl-Dickstein, Jason Phang, Jason Wei, Jason Yosinski, Jekaterina Novikova, Jelle Bosscher, Jennifer Marsh, Jeremy Kim, Jeroen Taal, Jesse H. Engel, Jesujoba Alabi, Jiacheng Xu, Jiaming Song, Jillian Tang, Joan Waweru, John Burden, John Miller, John U. Balis, Jonathan Batchelder, Jonathan Berant, Jörg Frohberg, Jos Rozen, José Hernández-Orallo, Joseph Boudeman, Joseph Guerr, Joseph Jones, Joshua B. Tenenbaum, Joshua S. Rule, Joyce Chua, Kamil Kanclerz, Karen Livescu, Karl Krauth, Karthik Gopalakrishnan, Katerina Ignatyeva, Katja Markert, Kaustubh D. Dhole, Kevin Gimpel, Kevin Omondi, Kory Mathewson, Kristen Chiafullo, Ksenia Shkaruta, Kumar Shridhar, Kyle McDonell, Kyle Richardson, Laria Reynolds, Leo Gao, Li Zhang, Liam Dugan, Lianhui Qin, Lidia Contreras Ochando, Louis-Philippe Morency, Luca Moschella, Lucas Lam, Lucy Noble, Ludwig Schmidt, Luheng He, Luis Oliveros Colón, Luke Metz, Lütfi Kerem Senel, Maarten Bosma, Maarten Sap, Maartje ter Hoeve, Maheen Farooqi, Manaal Faruqui, Mantas Mazeika, Marco Baturan, Marco Marelli, Marco Maru, María José Ramírez-Quintana, Marie Tolkiehn, Mario Giulianelli, Martha Lewis, Martin Potthast, Matthew L. Leavitt, Matthias Hagen, Mátyás Schubert, Medina Baitemirova, Melody Arnaud, Melvin McElrath, Michael A. Yee, Michael Cohen, Michael Gu, Michael I. Ivanitskiy, Michael Starritt, Michael Strube, Michal Swedrowski, Michele Bevilacqua, Michihiro Yasunaga, Mihir Kale, Mike Cain, Mimee Xu, Mirac Suzgun, Mitch Walker, Mo Tiwari, Mohit Bansal, Moin Aminnaseri, Mor Geva, Mozhdeh Gheini, Mukund Varma T., Nanyun Peng, Nathan A. Chi, Nayeon Lee, Neta Gur-Ari Krakover, Nicholas Cameron, Nicholas Roberts, Nick Doiron, Nicole Martinez, Nikita Nangia, Niklas Deckers, Niklas Muennighoff, Nitish Shirish Keskar, Niveditha Iyer, Noah Constant, Noah Fiedel, Nuan Wen, Oliver Zhang, Omar Agha, Omar Elbaghdadi, Omer Levy, Owain Evans, Pablo Antonio Moreno Casares, Parth Doshi, Pascale Fung, Paul Pu Liang, Paul Vicol, Pegah Alipoormolabashi, Peiyuan Liao, Percy Liang, Peter Chang, Peter Eckersley, Phu Mon Htut, Pinyu Hwang, Piotr Milkowski, Piyush Patil, Pouya Pezeshkpour, Priti Oli, Qiaozhu Mei, Qing Lyu, Qinlang Chen, Rabin Banjade, Rachel Etta Rudolph, Raefer Gabriel, Rahel Habacker, Ramon Risco, Raphaël Millière, Rhythm Garg, Richard Barnes, Rif A. Saurous, Riku Arakawa, Robbe Raymaekers, Robert Frank, Rohan Sikand, Roman Novak, Roman Sitelew, Ronan LeBras, Rosanne Liu, Rowan Jacobs, Rui Zhang, Ruslan Salakhutdinov, Ryan Chi, Ryan Lee, Ryan Stovall, Ryan Teehan, Rylan Yang, Sahib Singh, Saif M. Mohammad, Sajant Anand, Sam Dillavou, Sam Shleifer, Sam Wiseman, Samuel Gruetter, Samuel R. Bowman, Samuel S. Schoenholz, Sanghyun Han, Sanjeev Kwatra, Sarah A. Rous, Sarik Ghazarian, Sayan Ghosh, Sean Casey, Sebastian Bischoff, Sebastian Gehrmann, Sebastian Schuster, Sepideh Sadeghi, Shadi Hamdan, Sharon Zhou, Shashank Srivastava, Sherry Shi, Shikhar Singh, Shima Asaadi, Shixiang Shane Gu, Shubh Pachchigar, Shubham Toshniwal, Shyam Upadhyay, Shyamolima (Shammie) Debnath, Siamak Shakeri, Simon Thormeyer, Simone Melzi, Siva Reddy, Sneha Priscilla Makini, Soo-Hwan Lee, Spencer Torene, Sriharsha Hatwar, Stanislas Dehaene, Stefan Divic, Stefano Ermon, Stella Biderman, Stephanie Lin, Stephen Prasad, Steven T. Piantadosi, Stuart M. Shieber, Summer Misherghi, Svetlana Kiritchenko, Swaroop Mishra, Tal Linzen, Tal Schuster, Tao Li, Tao Yu, Tariq Ali, Tatsu Hashimoto, Te-Lin Wu, Théo Desbordes, Theodore Rothschild, Thomas Phan, Tianle Wang, Tiberius Nkinyili, Timo Schick, Timofei Kornev, Titus Tunduny, Tobias Gerstenberg, Trenton Chang, Trishala Neeraj, Tushar Khot, Tyler Shultz, Uri Shaham, Vedant Misra, Vera Demberg, Victoria Nyamai, Vikas Raunak, Vinay V. Ramasesh, Vinay Uday Prabhu, Vishakh Padmakumar, Vivek Srikumar, William Fedus, William Saunders, William Zhang, Wout Vossen, Xiang Ren, Xiaoyu Tong, Xinran Zhao, Xinyi Wu, Xudong Shen, Yadollah Yaghoobzadeh, Yair Lakretz, Yangqiu Song, Yasaman Bahri, Yejin Choi, Yichi Yang, Yiding Hao, Yifu Chen, Yonatan Belinkov, Yu Hou, Yufang Hou, Yuntao Bai, Zachary Seid, Zhuoye Zhao, Zijian Wang, Zijie J. Wang, Zirui Wang, Ziyi Wu
Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models. Trans. Mach. Learn. Res. 2023 (2023)
Oscar Li, James Harrison, Jascha Sohl-Dickstein, Virginia Smith, Luke Metz
Variance-Reduced Gradient Estimation via Noise-Reuse in Online Evolution Strategies. NeurIPS (2023)
Yilun Du, Conor Durkan, Robin Strudel, Joshua B. Tenenbaum, Sander Dieleman, Rob Fergus, Jascha Sohl-Dickstein, Arnaud Doucet, Will Sussman Grathwohl
Reduce, Reuse, Recycle: Compositional Generation with Energy-Based Diffusion Models and MCMC. ICML (2023)
Mitchell Wortsman, Peter J. Liu, Lechao Xiao, Katie Everett, Alex Alemi, Ben Adlam, John D. Co-Reyes, Izzeddin Gur, Abhishek Kumar, Roman Novak, Jeffrey Pennington, Jascha Sohl-Dickstein, Kelvin Xu, Jaehoon Lee, Justin Gilmer, Simon Kornblith
Small-scale proxies for large-scale Transformer training instabilities. CoRR (2023)
Meredith Ringel Morris, Jascha Sohl-Dickstein, Noah Fiedel, Tris Warkentin, Allan Dafoe, Aleksandra Faust, Clément Farabet, Shane Legg
Levels of AGI: Operationalizing Progress on the Path to AGI. CoRR (2023)
C. Daniel Freeman, Laura Culp, Aaron Parisi, Maxwell L. Bileschi, Gamaleldin F. Elsayed, Alex Rizkowsky, Isabelle Simpson, Alex Alemi, Azade Nova, Ben Adlam, Bernd Bohnet, Gaurav Mishra, Hanie Sedghi, Igor Mordatch, Izzeddin Gur, Jaehoon Lee, John D. Co-Reyes, Jeffrey Pennington, Kelvin Xu, Kevin Swersky, Kshiteej Mahajan, Lechao Xiao, Rosanne Liu, Simon Kornblith, Noah Constant, Peter J. Liu, Roman Novak, Yundi Qian, Noah Fiedel, Jascha Sohl-Dickstein
Frontier Language Models are not Robust to Adversarial Arithmetic, or "What do I need to say so you agree 2+2=5? CoRR (2023)
Luke Metz, C. Daniel Freeman, James Harrison, Niru Maheswaranathan, Jascha Sohl-Dickstein
Practical tradeoffs between memory, compute, and performance in learned optimizers. CoRR (2022)
Roman Novak, Jascha Sohl-Dickstein, Samuel S. Schoenholz
Fast Finite Width Neural Tangent Kernel. ICML (2022)
Luke Metz, C. Daniel Freeman, James Harrison, Niru Maheswaranathan, Jascha Sohl-Dickstein
Practical Tradeoffs between Memory, Compute, and Performance in Learned Optimizers. CoLLAs (2022)
Jiri Hron, Roman Novak, Jeffrey Pennington, Jascha Sohl-Dickstein
Wide Bayesian neural networks have a simple weight posterior: theory and accelerated sampling. ICML (2022)
Louis Kirsch, James Harrison, Jascha Sohl-Dickstein, Luke Metz
General-Purpose In-Context Learning by Meta-Learning Transformers. CoRR (2022)
Luke Metz, James Harrison, C. Daniel Freeman, Amil Merchant, Lucas Beyer, James Bradbury, Naman Agrawal, Ben Poole, Igor Mordatch, Adam Roberts, Jascha Sohl-Dickstein
VeLO: Training Versatile Learned Optimizers by Scaling Up. CoRR (2022)
David Dohan, Winnie Xu, Aitor Lewkowycz, Jacob Austin, David Bieber, Raphael Gontijo Lopes, Yuhuai Wu, Henryk Michalewski, Rif A. Saurous, Jascha Sohl-Dickstein, Kevin Murphy, Charles Sutton
Language Model Cascades. CoRR (2022)
Paul Vicol, Luke Metz, Jascha Sohl-Dickstein
Unbiased Gradient Estimation in Unrolled Computation Graphs with Persistent Evolution Strategies (Extended Abstract). IJCAI (2022)
Jiri Hron, Roman Novak, Jeffrey Pennington, Jascha Sohl-Dickstein
Wide Bayesian neural networks have a simple weight posterior: theory and accelerated sampling. CoRR (2022)
James Harrison, Luke Metz, Jascha Sohl-Dickstein
A Closer Look at Learned Optimization: Stability, Robustness, and Inductive Biases. NeurIPS (2022)
James Harrison, Luke Metz, Jascha Sohl-Dickstein
A Closer Look at Learned Optimization: Stability, Robustness, and Inductive Biases. CoRR (2022)
Roman Novak, Jascha Sohl-Dickstein, Samuel S. Schoenholz
Fast Finite Width Neural Tangent Kernel. CoRR (2022)
Kaustubh D. Dhole, Varun Gangal, Sebastian Gehrmann, Aadesh Gupta, Zhenhao Li, Saad Mahamood, Abinaya Mahendiran, Simon Mille, Ashish Srivastava, Samson Tan, Tongshuang Wu, Jascha Sohl-Dickstein, Jinho D. Choi, Eduard H. Hovy, Ondrej Dusek, Sebastian Ruder, Sajant Anand, Nagender Aneja, Rabin Banjade, Lisa Barthe, Hanna Behnke, Ian Berlot-Attwell, Connor Boyle, Caroline Brun, Marco Antonio Sobrevilla Cabezudo, Samuel Cahyawijaya, Emile Chapuis, Wanxiang Che, Mukund Choudhary, Christian Clauss, Pierre Colombo, Filip Cornell, Gautier Dagan, Mayukh Das, Tanay Dixit, Thomas Dopierre, Paul-Alexis Dray, Suchitra Dubey, Tatiana Ekeinhor, Marco Di Giovanni, Rishabh Gupta, Rishabh Gupta, Louanes Hamla, Sang Han, Fabrice Harel-Canada, Antoine Honore, Ishan Jindal, Przemyslaw K. Joniak, Denis Kleyko, Venelin Kovatchev, et al.
NL-Augmenter: A Framework for Task-Sensitive Natural Language Augmentation. CoRR (2021)
James Martens, Andy Ballard, Guillaume Desjardins, Grzegorz Swirszcz, Valentin Dalibard, Jascha Sohl-Dickstein, Samuel S. Schoenholz
Rapid training of deep neural networks without skip connections or normalization layers using Deep Kernel Shaping. CoRR (2021)
Neha S. Wadia, Daniel Duckworth, Samuel S. Schoenholz, Ethan Dyer, Jascha Sohl-Dickstein
Whitening and Second Order Optimization Both Make Information in the Dataset Unusable During Training, and Can Reduce or Prevent Generalization. ICML (2021)
Paul Vicol, Luke Metz, Jascha Sohl-Dickstein
Unbiased Gradient Estimation in Unrolled Computation Graphs with Persistent Evolution Strategies. ICML (2021)
Yang Song, Jascha Sohl-Dickstein, Diederik P. Kingma, Abhishek Kumar, Stefano Ermon, Ben Poole
Score-Based Generative Modeling through Stochastic Differential Equations. ICLR (2021)
Luke Metz, C. Daniel Freeman, Niru Maheswaranathan, Jascha Sohl-Dickstein
Training Learned Optimizers with Randomly Initialized Learned Optimizers. CoRR (2021)
Niru Maheswaranathan, David Sussillo, Luke Metz, Ruoxi Sun, Jascha Sohl-Dickstein
Reverse engineering learned optimizers reveals known and novel mechanisms. NeurIPS (2021)
Paul Vicol, Luke Metz, Jascha Sohl-Dickstein
Unbiased Gradient Estimation in Unrolled Computation Graphs with Persistent Evolution Strategies. CoRR (2021)
Yang Song, Jascha Sohl-Dickstein, Diederik P. Kingma, Abhishek Kumar, Stefano Ermon, Ben Poole
Score-Based Generative Modeling through Stochastic Differential Equations. CoRR (2020)
Tong Che, Ruixiang Zhang, Jascha Sohl-Dickstein, Hugo Larochelle, Liam Paull, Yuan Cao, Yoshua Bengio
Your GAN is Secretly an Energy-based Model and You Should use Discriminator Driven Latent Sampling. CoRR (2020)
Jaehoon Lee, Samuel S. Schoenholz, Jeffrey Pennington, Ben Adlam, Lechao Xiao, Roman Novak, Jascha Sohl-Dickstein
Finite Versus Infinite Neural Networks: an Empirical Study. NeurIPS (2020)
Luke Metz, Niru Maheswaranathan, C. Daniel Freeman, Ben Poole, Jascha Sohl-Dickstein
Tasks, stability, architecture, and compute: Training more effective learned optimizers, and using them to train themselves. CoRR (2020)
Jascha Sohl-Dickstein, Peter Battaglino, Michael Robert DeWeese
A new method for parameter estimation in probabilistic models: Minimum probability flow. CoRR (2020)
Luke Metz, Niru Maheswaranathan, Ruoxi Sun, C. Daniel Freeman, Ben Poole, Jascha Sohl-Dickstein
Using a thousand optimization tasks to learn hyperparameter search strategies. CoRR (2020)
Tong Che, Ruixiang Zhang, Jascha Sohl-Dickstein, Hugo Larochelle, Liam Paull, Yuan Cao, Yoshua Bengio
Your GAN is Secretly an Energy-based Model and You Should Use Discriminator Driven Latent Sampling. NeurIPS (2020)
Michael Laskin, Luke Metz, Seth Nabarrao, Mark Saroufim, Badreddine Noune, Carlo Luschi, Jascha Sohl-Dickstein, Pieter Abbeel
Parallel Training of Deep Networks with Local Updates. CoRR (2020)
Vinay Rao, Jascha Sohl-Dickstein
Is Batch Norm unique? An empirical investigation and prescription to emulate the best properties of common normalizers without batch dependence. CoRR (2020)
Neha S. Wadia, Daniel Duckworth, Samuel S. Schoenholz, Ethan Dyer, Jascha Sohl-Dickstein
Whitening and second order optimization both destroy information about the dataset, and can make generalization impossible. CoRR (2020)
Niru Maheswaranathan, David Sussillo, Luke Metz, Ruoxi Sun, Jascha Sohl-Dickstein
Reverse engineering learned optimizers reveals known and novel mechanisms. CoRR (2020)
Roman Novak, Lechao Xiao, Jiri Hron, Jaehoon Lee, Alexander A. Alemi, Jascha Sohl-Dickstein, Samuel S. Schoenholz
Neural Tangents: Fast and Easy Infinite Neural Networks in Python. ICLR (2020)
Jiri Hron, Yasaman Bahri, Jascha Sohl-Dickstein, Roman Novak
Infinite attention: NNGP and NTK for deep attention networks. CoRR (2020)
Jiri Hron, Yasaman Bahri, Jascha Sohl-Dickstein, Roman Novak
Infinite attention: NNGP and NTK for deep attention networks. ICML (2020)
Jascha Sohl-Dickstein, Roman Novak, Samuel S. Schoenholz, Jaehoon Lee
On the infinite width limit of neural networks with a standard parameterization. CoRR (2020)
Jaehoon Lee, Samuel S. Schoenholz, Jeffrey Pennington, Ben Adlam, Lechao Xiao, Roman Novak, Jascha Sohl-Dickstein
Finite Versus Infinite Neural Networks: an Empirical Study. CoRR (2020)
Aitor Lewkowycz, Yasaman Bahri, Ethan Dyer, Jascha Sohl-Dickstein, Guy Gur-Ari
The large learning rate phase of deep learning: the catapult mechanism. CoRR (2020)
Daniel S. Park, Jaehoon Lee, Daiyi Peng, Yuan Cao, Jascha Sohl-Dickstein
Towards NNGP-guided Neural Architecture Search. CoRR (2020)
Jiri Hron, Yasaman Bahri, Roman Novak, Jeffrey Pennington, Jascha Sohl-Dickstein
Exact posterior distributions of wide Bayesian neural networks. CoRR (2020)
Roman Novak, Lechao Xiao, Yasaman Bahri, Jaehoon Lee, Greg Yang, Jiri Hron, Daniel A. Abolafia, Jeffrey Pennington, Jascha Sohl-Dickstein
Bayesian Deep Convolutional Networks with Many Channels are Gaussian Processes. ICLR (Poster) (2019)
Jascha Sohl-Dickstein, Kenji Kawaguchi
Eliminating all bad Local Minima from Loss Landscapes without even adding an Extra Unit. CoRR (2019)
Jaehoon Lee, Lechao Xiao, Samuel S. Schoenholz, Yasaman Bahri, Jascha Sohl-Dickstein, Jeffrey Pennington
Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descent. CoRR (2019)
Luke Metz, Niru Maheswaranathan, Jeremy Nixon, C. Daniel Freeman, Jascha Sohl-Dickstein
Understanding and correcting pathologies in the training of learned optimizers. ICML (2019)
Stephan Hoyer, Jascha Sohl-Dickstein, Sam Greydanus
Neural reparameterization improves structural optimization. CoRR (2019)
Mahdi Karami, Dale Schuurmans, Jascha Sohl-Dickstein, Laurent Dinh, Daniel Duckworth
Invertible Convolutional Flow. NeurIPS (2019)
Laurent Dinh, Jascha Sohl-Dickstein, Razvan Pascanu, Hugo Larochelle
A RAD approach to deep mixture models. DGS@ICLR (2019)
Daniel S. Park, Jascha Sohl-Dickstein, Quoc V. Le, Samuel L. Smith
The Effect of Network Width on Stochastic Gradient Descent and Generalization: an Empirical Study. CoRR (2019)
Gamaleldin F. Elsayed, Ian J. Goodfellow, Jascha Sohl-Dickstein
Adversarial Reprogramming of Neural Networks. ICLR (Poster) (2019)
Luke Metz, Niru Maheswaranathan, Jonathon Shlens, Jascha Sohl-Dickstein, Ekin D. Cubuk
Using learned optimizers to make models robust to input noise. CoRR (2019)
Greg Yang, Jeffrey Pennington, Vinay Rao, Jascha Sohl-Dickstein, Samuel S. Schoenholz
A Mean Field Theory of Batch Normalization. ICLR (Poster) (2019)
Jaehoon Lee, Lechao Xiao, Samuel S. Schoenholz, Yasaman Bahri, Roman Novak, Jascha Sohl-Dickstein, Jeffrey Pennington
Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descent. NeurIPS (2019)
Greg Yang, Jeffrey Pennington, Vinay Rao, Jascha Sohl-Dickstein, Samuel S. Schoenholz
A Mean Field Theory of Batch Normalization. CoRR (2019)
Daniel S. Park, Jascha Sohl-Dickstein, Quoc V. Le, Samuel L. Smith
The Effect of Network Width on Stochastic Gradient Descent and Generalization: an Empirical Study. ICML (2019)
Niru Maheswaranathan, Luke Metz, George Tucker, Dami Choi, Jascha Sohl-Dickstein
Guided evolutionary strategies: augmenting random search with surrogate gradients. ICML (2019)
Christopher J. Shallue, Jaehoon Lee, Joseph M. Antognini, Jascha Sohl-Dickstein, Roy Frostig, George E. Dahl
Measuring the Effects of Data Parallelism on Neural Network Training. J. Mach. Learn. Res. 20 (2019)
Luke Metz, Niru Maheswaranathan, Brian Cheung, Jascha Sohl-Dickstein
Meta-Learning Update Rules for Unsupervised Representation Learning. ICLR (2019)
Roman Novak, Lechao Xiao, Jiri Hron, Jaehoon Lee, Alexander A. Alemi, Jascha Sohl-Dickstein, Samuel S. Schoenholz
Neural Tangents: Fast and Easy Infinite Neural Networks in Python. CoRR (2019)
Laurent Dinh, Jascha Sohl-Dickstein, Razvan Pascanu, Hugo Larochelle
A RAD approach to deep mixture models. CoRR (2019)
Luke Metz, Niru Maheswaranathan, Brian Cheung, Jascha Sohl-Dickstein
Learning Unsupervised Learning Rules. CoRR (2018)
Gamaleldin F. Elsayed, Shreya Shankar, Brian Cheung, Nicolas Papernot, Alexey Kurakin, Ian J. Goodfellow, Jascha Sohl-Dickstein
Adversarial Examples that Fool both Computer Vision and Time-Limited Humans. NeurIPS (2018)
Daniel Levy, Matthew D. Hoffman, Jascha Sohl-Dickstein
Generalizing Hamiltonian Monte Carlo with Neural Networks. ICLR (Poster) (2018)
Roman Novak, Yasaman Bahri, Daniel A. Abolafia, Jeffrey Pennington, Jascha Sohl-Dickstein
Sensitivity and Generalization in Neural Networks: an Empirical Study. ICLR (Poster) (2018)
Gamaleldin F. Elsayed, Shreya Shankar, Brian Cheung, Nicolas Papernot, Alex Kurakin, Ian J. Goodfellow, Jascha Sohl-Dickstein
Adversarial Examples that Fool both Human and Computer Vision. CoRR (2018)
Luke Metz, Niru Maheswaranathan, Jeremy Nixon, C. Daniel Freeman, Jascha Sohl-Dickstein
Learned optimizers that outperform SGD on wall-clock and test loss. CoRR (2018)
Lechao Xiao, Yasaman Bahri, Jascha Sohl-Dickstein, Samuel S. Schoenholz, Jeffrey Pennington
Dynamical Isometry and a Mean Field Theory of CNNs: How to Train 10, 000-Layer Vanilla Convolutional Neural Networks. CoRR (2018)
Niru Maheswaranathan, Luke Metz, George Tucker, Jascha Sohl-Dickstein
Guided evolutionary strategies: escaping the curse of dimensionality in random search. CoRR (2018)
Joseph M. Antognini, Jascha Sohl-Dickstein
PCA of high dimensional random walks with comparison to neural network training. CoRR (2018)
Jaehoon Lee, Yasaman Bahri, Roman Novak, Samuel S. Schoenholz, Jeffrey Pennington, Jascha Sohl-Dickstein
Deep Neural Networks as Gaussian Processes. ICLR (Poster) (2018)
Joseph M. Antognini, Jascha Sohl-Dickstein
PCA of high dimensional random walks with comparison to neural network training. NeurIPS (2018)
Lechao Xiao, Yasaman Bahri, Jascha Sohl-Dickstein, Samuel S. Schoenholz, Jeffrey Pennington
Dynamical Isometry and a Mean Field Theory of CNNs: How to Train 10, 000-Layer Vanilla Convolutional Neural Networks. ICML (2018)
Christopher J. Shallue, Jaehoon Lee, Joseph M. Antognini, Jascha Sohl-Dickstein, Roy Frostig, George E. Dahl
Measuring the Effects of Data Parallelism on Neural Network Training. CoRR (2018)
Roman Novak, Lechao Xiao, Jaehoon Lee, Yasaman Bahri, Daniel A. Abolafia, Jeffrey Pennington, Jascha Sohl-Dickstein
Bayesian Convolutional Neural Networks with Many Channels are Gaussian Processes. CoRR (2018)
Roman Novak, Yasaman Bahri, Daniel A. Abolafia, Jeffrey Pennington, Jascha Sohl-Dickstein
Sensitivity and Generalization in Neural Networks: an Empirical Study. CoRR (2018)
Samuel L. Smith, Daniel Duckworth, Quoc V. Le, Jascha Sohl-Dickstein
Stochastic natural gradient descent draws posterior samples in function space. CoRR (2018)
Gamaleldin F. Elsayed, Ian J. Goodfellow, Jascha Sohl-Dickstein
Adversarial Reprogramming of Neural Networks. CoRR (2018)
Luke Metz, Niru Maheswaranathan, Brian Cheung, Jascha Sohl-Dickstein
Learning to Learn Without Labels. ICLR (Workshop) (2018)
Samuel S. Schoenholz, Justin Gilmer, Surya Ganguli, Jascha Sohl-Dickstein
Deep Information Propagation. ICLR (Poster) (2017)
Jaehoon Lee, Yasaman Bahri, Roman Novak, Samuel S. Schoenholz, Jeffrey Pennington, Jascha Sohl-Dickstein
Deep Neural Networks as Gaussian Processes. CoRR (2017)
Olga Wichrowska, Niru Maheswaranathan, Matthew W. Hoffman, Sergio Gomez Colmenarejo, Misha Denil, Nando de Freitas, Jascha Sohl-Dickstein
Learned Optimizers that Scale and Generalize. ICML (2017)
Badr F. Albanna, Christopher Hillar, Jascha Sohl-Dickstein, Michael Robert DeWeese
Minimum and Maximum Entropy Distributions for Binary Systems with Known Means and Pairwise Correlations. Entropy 19 (8) (2017)
George Tucker, Andriy Mnih, Chris J. Maddison, Dieterich Lawson, Jascha Sohl-Dickstein
REBAR: Low-variance, unbiased gradient estimates for discrete latent variable models. NIPS (2017)
Jasmine Collins, Jascha Sohl-Dickstein, David Sussillo
Capacity and Trainability in Recurrent Neural Networks. ICLR (Poster) (2017)
Maithra Raghu, Ben Poole, Jon M. Kleinberg, Surya Ganguli, Jascha Sohl-Dickstein
On the Expressive Power of Deep Neural Networks. ICML (2017)
Samuel S. Schoenholz, Jeffrey Pennington, Jascha Sohl-Dickstein
A Correspondence Between Random Neural Networks and Statistical Field Theory. CoRR (2017)
Jakob N. Foerster, Justin Gilmer, Jascha Sohl-Dickstein, Jan Chorowski, David Sussillo
Input Switched Affine Networks: An RNN Architecture Designed for Interpretability. ICML (2017)
Daniel Levy, Matthew D. Hoffman, Jascha Sohl-Dickstein
Generalizing Hamiltonian Monte Carlo with Neural Networks. CoRR (2017)

Jascha Sohl-Dickstein.css-eqlu8i{font-size:small;padding-left:4px;}

Publication Activity (10 Years)

Top Topics

Top Venues

Jascha Sohl-Dickstein