Rohan Anil

Publication Activity (10 Years)

Years Active: 2016-2024
Publications (10 Years): 43

Top Topics

Bregman Divergences

Memory Efficient

Top Venues

Publications

Andrew Hard, Antonious M. Girgis, Ehsan Amid, Sean Augenstein, Lara McConnaughey, Rajiv Mathews, Rohan Anil
Learning from straggler clients in federated learning. CoRR (2024)
Sai Surya Duvvuri, Devvrit, Rohan Anil, Cho-Jui Hsieh, Inderjit S. Dhillon
Combining Axes Preconditioners through Kronecker Approximation for Deep Learning. ICLR (2024)
Machel Reid, Nikolay Savinov, Denis Teplyashin, Dmitry Lepikhin, Timothy P. Lillicrap, Jean-Baptiste Alayrac, Radu Soricut, Angeliki Lazaridou, Orhan Firat, Julian Schrittwieser, Ioannis Antonoglou, Rohan Anil, Sebastian Borgeaud, Andrew M. Dai, Katie Millican, Ethan Dyer, Mia Glaese, Thibault Sottiaux, Benjamin Lee, Fabio Viola, Malcolm Reynolds, Yuanzhong Xu, James Molloy, Jilin Chen, Michael Isard, Paul Barham, Tom Hennigan, Ross McIlroy, Melvin Johnson, Johan Schalkwyk, Eli Collins, Eliza Rutherford, Erica Moreira, Kareem Ayoub, Megha Goel, Clemens Meyer, Gregory Thornton, Zhen Yang, Henryk Michalewski, Zaheer Abbas, Nathan Schucher, Ankesh Anand, Richard Ives, James Keeling, Karel Lenc, Salem Haykal, Siamak Shakeri, Pranav Shyam, Aakanksha Chowdhery, Roman Ring, Stephen Spencer, Eren Sezener, et al.
Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. CoRR (2024)
Rohan Anil, Sebastian Borgeaud, Yonghui Wu, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M. Dai, Anja Hauth, Katie Millican, David Silver, Slav Petrov, Melvin Johnson, Ioannis Antonoglou, Julian Schrittwieser, Amelia Glaese, Jilin Chen, Emily Pitler, Timothy P. Lillicrap, Angeliki Lazaridou, Orhan Firat, James Molloy, Michael Isard, Paul Ronald Barham, Tom Hennigan, Benjamin Lee, Fabio Viola, Malcolm Reynolds, Yuanzhong Xu, Ryan Doherty, Eli Collins, Clemens Meyer, Eliza Rutherford, Erica Moreira, Kareem Ayoub, Megha Goel, George Tucker, Enrique Piqueras, Maxim Krikun, Iain Barr, Nikolay Savinov, Ivo Danihelka, Becca Roelofs, Anaïs White, Anders Andreassen, Tamara von Glehn, Lakshman Yagati, Mehran Kazemi, Lucas Gonzalez, Misha Khalman, Jakub Sygnowski, et al.
Gemini: A Family of Highly Capable Multimodal Models. CoRR (2023)
Fnu Devvrit, Sai Surya Duvvuri, Rohan Anil, Vineet Gupta, Cho-Jui Hsieh, Inderjit S. Dhillon
A Computationally Efficient Sparsified Online Newton Method. CoRR (2023)
Jared Lichtarge, Ehsan Amid, Shankar Kumar, Tien-Ju Yang, Rohan Anil, Rajiv Mathews
Heterogeneous Federated Learning Using Knowledge Codistillation. CoRR (2023)
Fnu Devvrit, Sai Surya Duvvuri, Rohan Anil, Vineet Gupta, Cho-Jui Hsieh, Inderjit S. Dhillon
A Computationally Efficient Sparsified Online Newton Method. NeurIPS (2023)
Vladimir Feinberg, Xinyi Chen, Y. Jennifer Sun, Rohan Anil, Elad Hazan
Sketchy: Memory-efficient Adaptive Regularization with Frequent Directions. CoRR (2023)
Ehsan Amid, Rohan Anil, Christopher Fifty, Manfred K. Warmuth
Layerwise Bregman Representation Learning of Neural Networks with Applications to Knowledge Distillation. Trans. Mach. Learn. Res. 2023 (2023)
George E. Dahl, Frank Schneider, Zachary Nado, Naman Agarwal, Chandramouli Shama Sastry, Philipp Hennig, Sourabh Medapati, Runa Eschenhagen, Priya Kasimbeg, Daniel Suo, Juhan Bae, Justin Gilmer, Abel L. Peirson, Bilal Khan, Rohan Anil, Mike Rabbat, Shankar Krishnan, Daniel Snider, Ehsan Amid, Kongtao Chen, Chris J. Maddison, Rakshith Vasudev, Michal Badura, Ankush Garg, Peter Mattson
Benchmarking Neural Network Training Algorithms. CoRR (2023)
Rohan Anil, Andrew M. Dai, Orhan Firat, Melvin Johnson, Dmitry Lepikhin, Alexandre Passos, Siamak Shakeri, Emanuel Taropa, Paige Bailey, Zhifeng Chen, Eric Chu, Jonathan H. Clark, Laurent El Shafey, Yanping Huang, Kathy Meier-Hellstern, Gaurav Mishra, Erica Moreira, Mark Omernick, Kevin Robinson, Sebastian Ruder, Yi Tay, Kefan Xiao, Yuanzhong Xu, Yujing Zhang, Gustavo Hernández Ábrego, Junwhan Ahn, Jacob Austin, Paul Barham, Jan A. Botha, James Bradbury, Siddhartha Brahma, Kevin Brooks, Michele Catasta, Yong Cheng, Colin Cherry, Christopher A. Choquette-Choo, Aakanksha Chowdhery, Clément Crepy, Shachi Dave, Mostafa Dehghani, Sunipa Dev, Jacob Devlin, Mark Díaz, Nan Du, Ethan Dyer, Vladimir Feinberg, Fangxiaoyu Feng, Vlad Fienber, Markus Freitag, Xavier Garcia, Sebastian Gehrmann, Lucas Gonzalez, et al.
PaLM 2 Technical Report. CoRR (2023)
Vladimir Feinberg, Xinyi Chen, Y. Jennifer Sun, Rohan Anil, Elad Hazan
Sketchy: Memory-efficient Adaptive Regularization with Frequent Directions. NeurIPS (2023)
Rohan Anil, Sandra Gadanho, Da Huang, Nijith Jacob, Zhuoshu Li, Dong Lin, Todd Phillips, Cristina Pop, Kevin Regan, Gil I. Shamir, Rakesh Shivanna, Qiqi Yan
On the Factory Floor: ML Engineering for Industrial-Scale Ads Recommendation Models. ORSUM@RecSys (2022)
Ehsan Amid, Rohan Anil, Christopher Fifty, Manfred K. Warmuth
Layerwise Bregman Representation Learning with Applications to Knowledge Distillation. CoRR (2022)
Rohan Anil, Badih Ghazi, Vineet Gupta, Ravi Kumar, Pasin Manurangsi
Large-Scale Differentially Private BERT. EMNLP (Findings) (2022)
Rohan Anil, Sandra Gadanho, Da Huang, Nijith Jacob, Zhuoshu Li, Dong Lin, Todd Phillips, Cristina Pop, Kevin Regan, Gil I. Shamir, Rakesh Shivanna, Qiqi Yan
On the Factory Floor: ML Engineering for Industrial-Scale Ads Recommendation Models. CoRR (2022)
Ehsan Amid, Rohan Anil, Manfred K. Warmuth
LocoProp: Enhancing BackProp via Local Loss Optimization. AISTATS (2022)
Ehsan Amid, Rohan Anil, Christopher Fifty, Manfred K. Warmuth
Step-size Adaptation Using Exponentiated Gradient Updates. CoRR (2022)
Ehsan Amid, Rohan Anil, Wojciech Kotlowski, Manfred K. Warmuth
Learning from Randomly Initialized Neural Network Features. CoRR (2022)
Lucas Beyer, Xiaohua Zhai, Amélie Royer, Larisa Markeeva, Rohan Anil, Alexander Kolesnikov
Knowledge distillation: A good teacher is patient and consistent. CVPR (2022)
Aurko Roy, Rohan Anil, Guangda Lai, Benjamin Lee, Jeffrey Zhao, Shuyuan Zhang, Shibo Wang, Ye Zhang, Shen Wu, Rigel Swavely, Tao Yu, Phuong Dao, Christopher Fifty, Zhifeng Chen, Yonghui Wu
N-Grammer: Augmenting Transformers with latent n-grams. CoRR (2022)
Chris Fifty, Ehsan Amid, Zhe Zhao, Tianhe Yu, Rohan Anil, Chelsea Finn
Efficiently Identifying Task Groupings for Multi-Task Learning. NeurIPS (2021)
Lucas Beyer, Xiaohua Zhai, Amélie Royer, Larisa Markeeva, Rohan Anil, Alexander Kolesnikov
Knowledge distillation: A good teacher is patient and consistent. CoRR (2021)
Ehsan Amid, Rohan Anil, Manfred K. Warmuth
LocoProp: Enhancing BackProp via Local Loss Optimization. CoRR (2021)
Zachary Nado, Justin Gilmer, Christopher J. Shallue, Rohan Anil, George E. Dahl
A Large Batch Optimizer Reality Check: Traditional, Generic Optimizers Suffice Across Batch Sizes. CoRR (2021)
Christopher Fifty, Ehsan Amid, Zhe Zhao, Tianhe Yu, Rohan Anil, Chelsea Finn
Efficiently Identifying Task Groupings for Multi-Task Learning. CoRR (2021)
Rohan Anil, Badih Ghazi, Vineet Gupta, Ravi Kumar, Pasin Manurangsi
Large-Scale Differentially Private BERT. CoRR (2021)
Christopher Fifty, Ehsan Amid, Zhe Zhao, Tianhe Yu, Rohan Anil, Chelsea Finn
Measuring and Harnessing Transference in Multi-Task Learning. CoRR (2020)
Naman Agarwal, Rohan Anil, Elad Hazan, Tomer Koren, Cyril Zhang
Disentangling Adaptive Gradient Methods from Learning Rates. CoRR (2020)
Naman Agarwal, Rohan Anil, Tomer Koren, Kunal Talwar, Cyril Zhang
Stochastic Optimization with Laggard Data Pipelines. CoRR (2020)
Naman Agarwal, Rohan Anil, Tomer Koren, Kunal Talwar, Cyril Zhang
Stochastic Optimization with Laggard Data Pipelines. NeurIPS (2020)
Rohan Anil, Vineet Gupta, Tomer Koren, Kevin Regan, Yoram Singer
Second Order Optimization Made Practical. CoRR (2020)
Ehsan Amid, Manfred K. Warmuth, Rohan Anil, Tomer Koren
Robust Bi-Tempered Logistic Loss Based on Bregman Divergences. NeurIPS (2019)
Rohan Anil, Vineet Gupta, Tomer Koren, Yoram Singer
Memory Efficient Adaptive Optimization. NeurIPS (2019)
Jonathan Shen, Patrick Nguyen, Yonghui Wu, Zhifeng Chen, Mia Xu Chen, Ye Jia, Anjuli Kannan, Tara N. Sainath, Yuan Cao, Chung-Cheng Chiu, Yanzhang He, Jan Chorowski, Smit Hinsu, Stella Laurenzo, James Qin, Orhan Firat, Wolfgang Macherey, Suyog Gupta, Ankur Bapna, Shuyuan Zhang, Ruoming Pang, Ron J. Weiss, Rohit Prabhavalkar, Qiao Liang, Benoit Jacob, Bowen Liang, HyoukJoong Lee, Ciprian Chelba, Sébastien Jean, Bo Li, Melvin Johnson, Rohan Anil, Rajat Tibrewal, Xiaobing Liu, Akiko Eriguchi, Navdeep Jaitly, Naveen Ari, Colin Cherry, Parisa Haghani, Otavio Good, Youlong Cheng, Raziel Alvarez, Isaac Caswell, Wei-Ning Hsu, Zongheng Yang, Kuan-Chieh Wang, Ekaterina Gonina, Katrin Tomanek, Ben Vanik, Zelin Wu, Llion Jones, Mike Schuster, Yanping Huang, Dehao Chen, Kazuki Irie, George F. Foster, John Richardson, Klaus Macherey, Antoine Bruguier, Heiga Zen, Colin Raffel, Shankar Kumar, Kanishka Rao, David Rybach, Matthew Murray, Vijayaditya Peddinti, Maxim Krikun, Michiel Bacchiani, Thomas B. Jablin, Robert Suderman, Ian Williams, Benjamin Lee, Deepti Bhatia, Justin Carlson, Semih Yavuz, Yu Zhang, Ian McGraw, Max Galkin, Qi Ge, Golan Pundak, Chad Whipkey, Todd Wang, Uri Alon, Dmitry Lepikhin, Ye Tian, Sara Sabour, William Chan, Shubham Toshniwal, Baohua Liao, Michael Nirschl, Pat Rondon
Lingvo: a Modular and Scalable Framework for Sequence-to-Sequence Modeling. CoRR (2019)
Rohan Anil, Vineet Gupta, Tomer Koren, Yoram Singer
Memory-Efficient Adaptive Optimization for Large-Scale Learning. CoRR (2019)
Rama Kumar Pasumarthi, Sebastian Bruch, Xuanhui Wang, Cheng Li, Michael Bendersky, Marc Najork, Jan Pfeifer, Nadav Golbandi, Rohan Anil, Stephan Wolf
TF-Ranking: Scalable TensorFlow Library for Learning-to-Rank. KDD (2019)
Ehsan Amid, Manfred K. Warmuth, Rohan Anil, Tomer Koren
Robust Bi-Tempered Logistic Loss Based on Bregman Divergences. CoRR (2019)
Rohan Anil, Gabriel Pereyra, Alexandre Passos, Róbert Ormándi, George E. Dahl, Geoffrey E. Hinton
Large scale distributed neural network training through online distillation. CoRR (2018)
Rohan Anil, Gabriel Pereyra, Alexandre Passos, Róbert Ormándi, George E. Dahl, Geoffrey E. Hinton
Large scale distributed neural network training through online distillation. ICLR (Poster) (2018)
Rama Kumar Pasumarthi, Xuanhui Wang, Cheng Li, Sebastian Bruch, Michael Bendersky, Marc Najork, Jan Pfeifer, Nadav Golbandi, Rohan Anil, Stephan Wolf
TF-Ranking: Scalable TensorFlow Library for Learning-to-Rank. CoRR (2018)
Heng-Tze Cheng, Levent Koc, Jeremiah Harmsen, Tal Shaked, Tushar Chandra, Hrishi Aradhye, Glen Anderson, Greg Corrado, Wei Chai, Mustafa Ispir, Rohan Anil, Zakaria Haque, Lichan Hong, Vihan Jain, Xiaobing Liu, Hemal Shah
Wide & Deep Learning for Recommender Systems. DLRS@RecSys (2016)
Heng-Tze Cheng, Levent Koc, Jeremiah Harmsen, Tal Shaked, Tushar Chandra, Hrishi Aradhye, Glen Anderson, Greg Corrado, Wei Chai, Mustafa Ispir, Rohan Anil, Zakaria Haque, Lichan Hong, Vihan Jain, Xiaobing Liu, Hemal Shah
Wide & Deep Learning for Recommender Systems. CoRR (2016)