Quentin Anthony

Publication Activity (10 Years)

Years Active: 2019-2024
Publications (10 Years): 36

Top Topics

Comparative Study

Parallel Tree Search

Top Venues

IPDPS Workshops

Publications

Paolo Glorioso, Quentin Anthony, Yury Tokpanov, James Whittington, Jonathan Pilault, Adam Ibrahim, Beren Millidge
Zamba: A Compact 7B SSM Hybrid Model. CoRR (2024)
Junqi Yin, Avishek Bose, Guojing Cong, Isaac Lyngaas, Quentin Anthony
Comparative Study of Large Language Model Architectures on Frontier. CoRR (2024)
Jinghan Yao, Quentin Anthony, Aamir Shafi, Hari Subramoni, Dhabaleswar K. Panda
Exploiting Inter-Layer Expert Affinity for Accelerating Mixture-of-Experts Model Inference. IPDPS (2024)
Bo Peng, Daniel Goldstein, Quentin Anthony, Alon Albalak, Eric Alcaide, Stella Biderman, Eugene Cheah, Xingjian Du, Teddy Ferdinan, Haowen Hou, Przemyslaw Kazienko, Kranthi Kiran GV, Jan Kocon, Bartlomiej Koptyra, Satyapriya Krishna, Ronald McClelland Jr., Niklas Muennighoff, Fares Obeid, Atsushi Saito, Guangyu Song, Haoqin Tu, Stanislaw Wozniak, Ruichong Zhang, Bingchen Zhao, Qihang Zhao, Peng Zhou, Jian Zhu, Rui-Jie Zhu
Eagle and Finch: RWKV with Matrix-Valued States and Dynamic Recurrence. CoRR (2024)
Junqi Yin, Avishek Bose, Guojing Cong, Isaac Lyngaas, Quentin Anthony
Comparative Study of Large Language Model Architectures on Frontier. IPDPS (2024)
Radha Gulhane, Quentin Anthony, Aamir Shafi, Hari Subramoni, Dhabaleswar K. Panda
Infer-HiRes: Accelerating Inference for High-Resolution Images with Quantization and Distributed Deep Learning. PEARC (2024)
Quentin Anthony, Jacob Hatef, Deepak Narayanan, Stella Biderman, Stas Bekman, Junqi Yin, Aamir Shafi, Hari Subramoni, Dhabaleswar K. Panda
The Case for Co-Designing Model Architectures with Hardware. ICPP (2024)
Yury Tokpanov, Beren Millidge, Paolo Glorioso, Jonathan Pilault, Adam Ibrahim, James Whittington, Quentin Anthony
Zyda: A 1.3T Dataset for Open Language Modeling. CoRR (2024)
Quentin Anthony, Yury Tokpanov, Paolo Glorioso, Beren Millidge
BlackMamba: Mixture of Experts for State-Space Models. CoRR (2024)
Jinghan Yao, Quentin Anthony, Aamir Shafi, Hari Subramoni, Dhabaleswar K. Panda
Exploiting Inter-Layer Expert Affinity for Accelerating Mixture-of-Experts Model Inference. CoRR (2024)
Quentin Anthony, Jacob Hatef, Deepak Narayanan, Stella Biderman, Stas Bekman, Junqi Yin, Aamir Shafi, Hari Subramoni, Dhabaleswar K. Panda
The Case for Co-Designing Model Architectures with Hardware. CoRR (2024)
Adam Ibrahim, Benjamin Thérien, Kshitij Gupta, Mats L. Richter, Quentin Anthony, Timothée Lesort, Eugene Belilovsky, Irina Rish
Simple and Scalable Strategies to Continually Pre-train Large Language Models. CoRR (2024)
Quentin Anthony, Ammar Ahmad Awan, Jeff Rasley, Yuxiong He, Aamir Shafi, Mustafa Abduljabbar, Hari Subramoni, Dhabaleswar K. Panda
MCR-DL: Mix-and-Match Communication Runtime for Deep Learning. CoRR (2023)
Alexander Havrilla, Maksym Zhuravinskyi, Duy Phung, Aman Tiwari, Jonathan Tow, Stella Biderman, Quentin Anthony, Louis Castricato
trlX: A Framework for Large Scale Reinforcement Learning from Human Feedback. EMNLP (2023)
Quentin Anthony, Ammar Ahmad Awan, Jeff Rasley, Yuxiong He, Aamir Shafi, Mustafa Abduljabbar, Hari Subramoni, Dhabaleswar K. Panda
MCR-DL: Mix-and-Match Communication Runtime for Deep Learning. IPDPS (2023)
Stella Biderman, USVSN Sai Prashanth, Lintang Sutawika, Hailey Schoelkopf, Quentin Anthony, Shivanshu Purohit, Edward Raff
Emergent and Predictable Memorization in Large Language Models. NeurIPS (2023)
Quentin Anthony, Lang Xu, Aamir Shafi, Hari Subramoni, Dhabaleswar K. Panda
ScaMP: Scalable Meta-Parallelism for Deep Learning Search. CCGrid (2023)
Stella Biderman, Hailey Schoelkopf, Quentin Anthony, Herbie Bradley, Kyle O'Brien, Eric Hallahan, Mohammad Aflah Khan, Shivanshu Purohit, USVSN Sai Prashanth, Edward Raff, Aviya Skowron, Lintang Sutawika, Oskar van der Wal
Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling. CoRR (2023)
Qinghua Zhou, Quentin Anthony, Lang Xu, Aamir Shafi, Mustafa Abduljabbar, Hari Subramoni, Dhabaleswar K. Panda
Accelerating Distributed Deep Learning Training with Compression Assisted Allgather and Reduce-Scatter Communication. IPDPS (2023)
Stella Biderman, USVSN Sai Prashanth, Lintang Sutawika, Hailey Schoelkopf, Quentin Anthony, Shivanshu Purohit, Edward Raff
Emergent and Predictable Memorization in Large Language Models. CoRR (2023)
Bo Peng, Eric Alcaide, Quentin Anthony, Alon Albalak, Samuel Arcadinho, Huanqi Cao, Xin Cheng, Michael Chung, Matteo Grella, Kranthi Kiran G. V., Xuzheng He, Haowen Hou, Przemyslaw Kazienko, Jan Kocon, Jiaming Kong, Bartlomiej Koptyra, Hayden Lau, Krishna Sri Ipsit Mantri, Ferdinand Mom, Atsushi Saito, Xiangru Tang, Bolun Wang, Johan S. Wind, Stanislaw Wozniak, Ruichong Zhang, Zhenyuan Zhang, Qihang Zhao, Peng Zhou, Jian Zhu, Rui-Jie Zhu
RWKV: Reinventing RNNs for the Transformer Era. CoRR (2023)
Bo Peng, Eric Alcaide, Quentin Anthony, Alon Albalak, Samuel Arcadinho, Stella Biderman, Huanqi Cao, Xin Cheng, Michael Chung, Leon Derczynski, Xingjian Du, Matteo Grella, Kranthi Gv, Xuzheng He, Haowen Hou, Przemyslaw Kazienko, Jan Kocon, Jiaming Kong, Bartlomiej Koptyra, Hayden Lau, Jiaju Lin, Krishna Sri Ipsit Mantri, Ferdinand Mom, Atsushi Saito, Guangyu Song, Xiangru Tang, Johan S. Wind, Stanislaw Wozniak, Zhenyuan Zhang, Qinghua Zhou, Jian Zhu, Rui-Jie Zhu
RWKV: Reinventing RNNs for the Transformer Era. EMNLP (Findings) (2023)
Kshitij Gupta, Benjamin Thérien, Adam Ibrahim, Mats L. Richter, Quentin Anthony, Eugene Belilovsky, Irina Rish, Timothée Lesort
Continual Pre-Training of Large Language Models: How to (re)warm your model? CoRR (2023)
Quentin Anthony, Lang Xu, Aamir Shafi, Hari Subramoni, Dhabaleswar K. Panda
ScaMP: Scalable Meta-Parallelism for Deep Learning Search. CCGridW (2023)
Sid Black, Stella Biderman, Eric Hallahan, Quentin Anthony, Leo Gao, Laurence Golding, Horace He, Connor Leahy, Kyle McDonell, Jason Phang, Michael Pieler, USVSN Sai Prashanth, Shivanshu Purohit, Laria Reynolds, Jonathan Tow, Ben Wang, Samuel Weinbach
GPT-NeoX-20B: An Open-Source Autoregressive Language Model. CoRR (2022)
Arpan Jain, Aamir Shafi, Quentin Anthony, Pouya Kousha, Hari Subramoni, Dhabaleswar K. Panda
Hy-Fi: Hybrid Five-Dimensional Parallel DNN Training on High-Performance GPU Clusters. ISC (2022)
Qinghua Zhou, Pouya Kousha, Quentin Anthony, Kawthar Shafie Khorassani, Aamir Shafi, Hari Subramoni, Dhabaleswar K. Panda
Accelerating MPI All-to-All Communication with Online Compression on Modern GPU Clusters. ISC (2022)
Qinghua Zhou, Quentin Anthony, Aamir Shafi, Hari Subramoni, Dhabaleswar K. Panda
Accelerating Broadcast Communication with GPU Compression for Deep Learning Workloads. HIPC (2022)
Quentin Anthony, Donglai Dai
Evaluating Multi-Level Checkpointing for Distributed Deep Neural Network Training. SC (Workshops) (2021)
Pouya Kousha, Quentin Anthony, Hari Subramoni, Dhabaleswar K. Panda
Cross-layer Visualization and Profiling of Network and I/O Communication for HPC Clusters. CoRR (2021)
Quentin Anthony, Lang Xu, Hari Subramoni, Dhabaleswar K. D. K. Panda
Scaling Single-Image Super-Resolution Training on Modern HPC Clusters: Early Experiences. IPDPS Workshops (2021)
Quentin Anthony, Ammar Ahmad Awan, Arpan Jain, Hari Subramoni, Dhabaleswar K. D. K. Panda
Efficient Training of Semantic Image Segmentation on Summit using Horovod and MVAPICH2-GDR. IPDPS Workshops (2020)
Seyedeh Mahdieh Ghazimirsaeed, Quentin Anthony, Aamir Shafi, Hari Subramoni, Dhabaleswar K. D. K. Panda
Accelerating GPU-based Machine Learning in Python using MPI Library: A Case Study with MVAPICH2-GDR. MLHPC/AI4S@SC (2020)
Ammar Ahmad Awan, Arpan Jain, Quentin Anthony, Hari Subramoni, Dhabaleswar K. Panda
HyPar-Flow: Exploiting MPI and Keras for Scalable Hybrid-Parallel DNN Training with TensorFlow. ISC (2020)
Ammar Ahmad Awan, Arpan Jain, Quentin Anthony, Hari Subramoni, Dhabaleswar K. Panda
HyPar-Flow: Exploiting MPI and Keras for Scalable Hybrid-Parallel DNN Training using TensorFlow. CoRR (2019)
Arpan Jain, Ammar Ahmad Awan, Quentin Anthony, Hari Subramoni, Dhabaleswar K. Panda
Performance Characterization of DNN Training using TensorFlow and PyTorch on Modern Clusters. CLUSTER (2019)