Minjia Zhang

Publication Activity (10 Years)

Years Active: 2010-2024
Publications (10 Years): 83

Top Topics

Episodic Memory

Efficient Inference

Nearest Neighbor Search

Top Venues

Publications

Xinyu Lian, Sam Ade Jacobs, Lev Kurilenko, Masahiro Tanaka, Stas Bekman, Olatunji Ruwase, Minjia Zhang
Universal Checkpointing: Efficient and Flexible Checkpointing for Large Scale Distributed Training. CoRR (2024)
Zheng Wang, Boxiao Jin, Zhongzhi Yu, Minjia Zhang
Model Tells You Where to Merge: Adaptive KV Cache Merging for LLMs on Long-Context Tasks. CoRR (2024)
Yongye Su, Yinqi Sun, Minjia Zhang, Jianguo Wang
Vexless: A Serverless Vector Data Management System Using Cloud Functions. Proc. ACM Manag. Data 2 (3) (2024)
Haozhe Zhao, Xiaojian Ma, Liang Chen, Shuzheng Si, Rujie Wu, Kaikai An, Peiyu Yu, Minjia Zhang, Qing Li, Baobao Chang
UltraEdit: Instruction-based Fine-Grained Image Editing at Scale. CoRR (2024)
Sam Ade Jacobs, Masahiro Tanaka, Chengming Zhang, Minjia Zhang, Reza Yazdani Aminabadi, Shuaiwen Leon Song, Samyam Rajbhandari, Yuxiong He
System Optimizations for Enabling Training of Extreme Long Sequence Transformer Models. IPDPS (Workshops) (2024)
Suyu Ge, Yunan Zhang, Liyuan Liu, Minjia Zhang, Jiawei Han, Jianfeng Gao
Model Tells You What to Discard: Adaptive KV Cache Compression for LLMs. ICLR (2024)
Jiangfei Duan, Ziang Song, Xupeng Miao, Xiaoli Xi, Dahua Lin, Harry Xu, Minjia Zhang, Zhihao Jia
Parcae: Proactive, Liveput-Optimized DNN Training on Preemptible Instances. NSDI (2024)
Sam Ade Jacobs, Masahiro Tanaka, Chengming Zhang, Minjia Zhang, Reza Yazdani Aminadabi, Shuaiwen Leon Song, Samyam Rajbhandari, Yuxiong He
System Optimizations for Enabling Training of Extreme Long Sequence Transformer Models. PODC (2024)
Conglong Li, Zhewei Yao, Xiaoxia Wu, Minjia Zhang, Connor Holmes, Cheng Li, Yuxiong He
DeepSpeed Data Efficiency: Improving Deep Learning Model Quality and Training Efficiency via Efficient Data Sampling and Routing. AAAI (2024)
Yao Lu, Song Bian, Lequn Chen, Yongjun He, Yulong Hui, Matthew Lentz, Beibin Li, Fei Liu, Jialin Li, Qi Liu, Rui Liu, Xiaoxuan Liu, Lin Ma, Kexin Rong, Jianguo Wang, Yingjun Wu, Yongji Wu, Huanchen Zhang, Minjia Zhang, Qizhen Zhang, Tianyi Zhou, Danyang Zhuo
Computing in the Era of Large Generative Models: From Cloud-Native to AI-Native. CoRR (2024)
Jiangfei Duan, Ziang Song, Xupeng Miao, Xiaoli Xi, Dahua Lin, Harry Xu, Minjia Zhang, Zhihao Jia
Parcae: Proactive, Liveput-Optimized DNN Training on Preemptible Instances. CoRR (2024)
Fengxiang Bie, Yibo Yang, Zhongzhu Zhou, Adam Ghanem, Minjia Zhang, Zhewei Yao, Xiaoxia Wu, Connor Holmes, Pareesa Ameneh Golnari, David A. Clifton, Yuxiong He, Dacheng Tao, Shuaiwen Leon Song
RenAIssance: A Survey into AI Text-to-Image Generation in the Era of Large Model. CoRR (2023)
Zhewei Yao, Xiaoxia Wu, Conglong Li, Minjia Zhang, Heyang Qin, Olatunji Ruwase, Ammar Ahmad Awan, Samyam Rajbhandari, Yuxiong He
DeepSpeed-VisualChat: Multi-Round Multi-Image Interleave Chat via Multi-Modal Causal Attention. CoRR (2023)
John Thorpe, Pengzhan Zhao, Jonathan Eyolfson, Yifan Qiao, Zhihao Jia, Minjia Zhang, Ravi Netravali, Guoqing Harry Xu
Bamboo: Making Preemptible Instances Resilient for Affordable Training of Large DNNs. NSDI (2023)
Zhewei Yao, Reza Yazdani Aminabadi, Olatunji Ruwase, Samyam Rajbhandari, Xiaoxia Wu, Ammar Ahmad Awan, Jeff Rasley, Minjia Zhang, Conglong Li, Connor Holmes, Zhongzhu Zhou, Michael Wyatt, Molly Smith, Lev Kurilenko, Heyang Qin, Masahiro Tanaka, Shuai Che, Shuaiwen Leon Song, Yuxiong He
DeepSpeed-Chat: Easy, Fast and Affordable RLHF Training of ChatGPT-like Models at All Scales. CoRR (2023)
Sam Ade Jacobs, Masahiro Tanaka, Chengming Zhang, Minjia Zhang, Shuaiwen Leon Song, Samyam Rajbhandari, Yuxiong He
DeepSpeed Ulysses: System Optimizations for Enabling Training of Extreme Long Sequence Transformer Models. CoRR (2023)
Yucheng Lu, Conglong Li, Minjia Zhang, Christopher De Sa, Yuxiong He
Maximizing Communication Efficiency for Large-scale Training via 0/1 Adam. ICLR (2023)
Minjia Zhang, Uma-Naresh Niranjan, Yuxiong He
Revisiting the Efficiency-Accuracy Tradeoff in Adapting Transformer Models via Adversarial Fine-Tuning. ECAI (2023)
Reza Yazdani Aminabadi, Olatunji Ruwase, Minjia Zhang, Yuxiong He, José-María Arnau, Antonio Gonazalez
SHARP: An Adaptable, Energy-Efficient Accelerator for Recurrent Neural Networks. ACM Trans. Embed. Comput. Syst. 22 (2) (2023)
Minjia Zhang, Jie Ren, Zhen Peng, Ruoming Jin, Dong Li, Bin Ren
iQAN: Fast and Accurate Vector Search with Efficient Intra-Query Parallelism on Multi-Core Architectures. IEEE Data Eng. Bull. 46 (3) (2023)
Xinyue Ma, Suyeon Jeong, Minjia Zhang, Di Wang, Jonghyun Choi, Myeongjae Jeon
Cost-effective On-device Continual Learning over Memory Hierarchy with Miro. MobiCom (2023)
Zhen Peng, Minjia Zhang, Kai Li, Ruoming Jin, Bin Ren
iQAN: Fast and Accurate Vector Search with Efficient Intra-Query Parallelism on Multi-Core Architectures. PPoPP (2023)
Suyu Ge, Yunan Zhang, Liyuan Liu, Minjia Zhang, Jiawei Han, Jianfeng Gao
Model Tells You What to Discard: Adaptive KV Cache Compression for LLMs. CoRR (2023)
Min Zhang, Fuxun Yu, Yongbo Yu, Minjia Zhang, Ang Li, Xiang Chen
FedHC: A Scalable Federated Learning Framework for Heterogeneous and Resource-Constrained Clients. CoRR (2023)
Xinyue Ma, Suyeon Jeong, Minjia Zhang, Di Wang, Jonghyun Choi, Myeongjae Jeon
Cost-effective On-device Continual Learning over Memory Hierarchy with Miro. CoRR (2023)
Shuangyan Yang, Minjia Zhang, Wenqian Dong, Dong Li
Betty: Enabling Large-Scale GNN Training with Batch-Level Graph Partitioning. ASPLOS (2) (2023)
Shuaiwen Leon Song, Bonnie Kruft, Minjia Zhang, Conglong Li, Shiyang Chen, Chengming Zhang, Masahiro Tanaka, Xiaoxia Wu, Jeff Rasley, Ammar Ahmad Awan, Connor Holmes, Martin Cai, Adam Ghanem, Zhongzhu Zhou, Yuxiong He, Pete Luferenko, Divya Kumar, Jonathan A. Weyn, Ruixiong Zhang, Sylwester Klocek, Volodymyr Vragov, Mohammed AlQuraishi, Gustaf Ahdritz, Christina Floristean, Cristina Negri, Rao Kotamarthi, Venkatram Vishwanath, Arvind Ramanathan, Sam Foreman, Kyle Hippe, Troy Arcomano, Romit Maulik, Maxim Zvyagin, Alexander Brace, Bin Zhang, Cindy Orozco Bohorquez, Austin Clyde, Bharat Kale, Danilo Perez-Rivera, Heng Ma, Carla M. Mann, Michael W. Irvin, J. Gregory Pauloski, Logan T. Ward, Valérie Hayot-Sasson, Murali Emani, Zhen Xie, Diangen Lin, Maulik Shukla, Ian T. Foster, James J. Davis, Michael E. Papka, Thomas S. Brettin, Prasanna Balaprakash, Gina Tourassi, John Gounley, Heidi A. Hanson, Thomas E. Potok, Massimiliano Lupo Pasini, Kate Evans, Dan Lu, Dalton D. Lunga, Junqi Yin, Sajal Dash, Feiyi Wang, Mallikarjun Shankar, Isaac Lyngaas, Xiao Wang, Guojing Cong, Pei Zhang, Ming Fan, Siyan Liu, Adolfy Hoisie, Shinjae Yoo, Yihui Ren, William Tang, Kyle Felker, Alexey Svyatkovskiy, Hang Liu, Ashwin M. Aji, Angela Dalton, Michael J. Schulte, Karl Schulz, Yuntian Deng, Weili Nie, Josh Romero, Christian Dallago, Arash Vahdat, Chaowei Xiao, Thomas Gibbs, Anima Anandkumar, Rick Stevens
DeepSpeed4Science Initiative: Enabling Large-Scale Scientific Discovery through Sophisticated AI System Technologies. CoRR (2023)
Xiaoxia Wu, Zhewei Yao, Minjia Zhang, Conglong Li, Yuxiong He
Extreme Compression for Pre-trained Transformers Made Simple and Efficient. CoRR (2022)
Samyam Rajbhandari, Conglong Li, Zhewei Yao, Minjia Zhang, Reza Yazdani Aminabadi, Ammar Ahmad Awan, Jeff Rasley, Yuxiong He
DeepSpeed-MoE: Advancing Mixture-of-Experts Inference and Training to Power Next-Generation AI Scale. ICML (2022)
Yucheng Lu, Conglong Li, Minjia Zhang, Christopher De Sa, Yuxiong He
Maximizing Communication Efficiency for Large-scale Training via 0/1 Adam. CoRR (2022)
John Thorpe, Pengzhan Zhao, Jonathan Eyolfson, Yifan Qiao, Zhihao Jia, Minjia Zhang, Ravi Netravali, Guoqing Harry Xu
Bamboo: Making Preemptible Instances Resilient for Affordable Training of Large DNNs. CoRR (2022)
Zhewei Yao, Xiaoxia Wu, Conglong Li, Connor Holmes, Minjia Zhang, Cheng Li, Yuxiong He
Random-LTD: Random and Layerwise Token Dropping Brings Efficient Training for Large-scale Transformers. CoRR (2022)
Minjia Zhang, Wenhan Wang, Yuxiong He
GraSP: Optimizing Graph-based Nearest Neighbor Search with Subgraph Sampling and Pruning. WSDM (2022)
Soobee Lee, Minindu Weerakoon, Jonghyun Choi, Minjia Zhang, Di Wang, Myeongjae Jeon
CarM: hierarchical episodic memory for continual learning. DAC (2022)
Zhewei Yao, Reza Yazdani Aminabadi, Minjia Zhang, Xiaoxia Wu, Conglong Li, Yuxiong He
ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformers. CoRR (2022)
Conglong Li, Zhewei Yao, Xiaoxia Wu, Minjia Zhang, Yuxiong He
DeepSpeed Data Efficiency: Improving Deep Learning Model Quality and Training Efficiency via Efficient Data Sampling and Routing. CoRR (2022)
Connor Holmes, Minjia Zhang, Yuxiong He, Bo Wu
Compressing Pre-trained Transformers via Low-Bit NxM Sparsity for Natural Language Understanding. CoRR (2022)
Xiaoxia Wu, Zhewei Yao, Minjia Zhang, Conglong Li, Yuxiong He
XTC: Extreme Compression for Pre-trained Transformers Made Simple and Efficient. NeurIPS (2022)
Conglong Li, Minjia Zhang, Yuxiong He
The Stability-Efficiency Dilemma: Investigating Sequence Length Warmup for Training GPT Models. NeurIPS (2022)
Yongbo Yu, Fuxun Yu, Zirui Xu, Di Wang, Minjia Zhang, Ang Li, Shawn Bray, Chenchen Liu, Xiang Chen
Powering Multi-Task Federated Learning with Competitive GPU Resource Sharing. WWW (Companion Volume) (2022)
Samyam Rajbhandari, Conglong Li, Zhewei Yao, Minjia Zhang, Reza Yazdani Aminabadi, Ammar Ahmad Awan, Jeff Rasley, Yuxiong He
DeepSpeed-MoE: Advancing Mixture-of-Experts Inference and Training to Power Next-Generation AI Scale. CoRR (2022)
Zhen Peng, Minjia Zhang, Kai Li, Ruoming Jin, Bin Ren
Speed-ANN: Low-Latency and High-Accuracy Nearest Neighbor Search via Intra-Query Parallelism. CoRR (2022)
Minjia Zhang, Uma-Naresh Niranjan, Yuxiong He
Adversarial Data Augmentation for Task-Specific Knowledge Distillation of Pre-trained Transformers. AAAI (2022)
Minjia Zhang, Uma-Naresh Niranjan, Yuxiong He
ScaLA: Accelerating Adaptation of Pre-Trained Transformer-Based Language Models via Efficient Large-Batch Adversarial Noise. CoRR (2022)
Fuxun Yu, Di Wang, Longfei Shangguan, Minjia Zhang, Chenchen Liu, Xiang Chen
A Survey of Multi-Tenant Deep Learning Inference on GPU. CoRR (2022)
Reza Yazdani Aminabadi, Samyam Rajbhandari, Ammar Ahmad Awan, Cheng Li, Du Li, Elton Zheng, Olatunji Ruwase, Shaden Smith, Minjia Zhang, Jeff Rasley, Yuxiong He
DeepSpeed- Inference: Enabling Efficient Inference of Transformer Models at Unprecedented Scale. SC (2022)
Reza Yazdani Aminabadi, Samyam Rajbhandari, Minjia Zhang, Ammar Ahmad Awan, Cheng Li, Du Li, Elton Zheng, Jeff Rasley, Shaden Smith, Olatunji Ruwase, Yuxiong He
DeepSpeed Inference: Enabling Efficient Inference of Transformer Models at Unprecedented Scale. CoRR (2022)
Zhewei Yao, Reza Yazdani Aminabadi, Minjia Zhang, Xiaoxia Wu, Conglong Li, Yuxiong He
ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformers. NeurIPS (2022)
Conglong Li, Minjia Zhang, Yuxiong He
Curriculum Learning: A Regularization Method for Efficient and Stable Billion-Scale GPT Model Pre-Training. CoRR (2021)
Minjia Zhang
DL Inference and Training Optimization Towards Speed and Scale. WWW (Companion Volume) (2021)
Fuxun Yu, Di Wang, Longfei Shangguan, Minjia Zhang, Xulong Tang, Chenchen Liu, Xiang Chen
A Survey of Large-Scale Deep Learning Serving System Optimization: Challenges and Opportunities. CoRR (2021)
Junfeng Zhao, Minjia Zhang, Hongji Yang
Vertical Scaling of Resource for OpenMP Application. ICSOC (2021)
Jie Ren, Samyam Rajbhandari, Reza Yazdani Aminabadi, Olatunji Ruwase, Shuangyan Yang, Minjia Zhang, Dong Li, Yuxiong He
ZeRO-Offload: Democratizing Billion-Scale Model Training. USENIX Annual Technical Conference (2021)
Minjia Zhang, Zehua Hu, Mingqin Li
DUET: A Compiler-Runtime Subgraph Scheduling Approach for Tensor Programs on a Coupled CPU-GPU Architecture. IPDPS (2021)
Jie Ren, Samyam Rajbhandari, Reza Yazdani Aminabadi, Olatunji Ruwase, Shuangyan Yang, Minjia Zhang, Dong Li, Yuxiong He
ZeRO-Offload: Democratizing Billion-Scale Model Training. CoRR (2021)
Dantong Zhu, Minjia Zhang
Understanding and Generalizing Monotonic Proximity Graphs for Approximate Nearest Neighbor Search. CoRR (2021)
Minjia Zhang, Menghao Li, Chi Wang, Mingqin Li
DynaTune: Dynamic Tensor Program Optimization in Deep Neural Network Compilation. ICLR (2021)
Soobee Lee, Minindu Weerakoon, Jonghyun Choi, Minjia Zhang, Di Wang, Myeongjae Jeon
Carousel Memory: Rethinking the Design of Episodic Memory for Continual Learning. CoRR (2021)
Jie Ren, Jiaolin Luo, Kai Wu, Minjia Zhang, Hyeran Jeon, Dong Li
Sentinel: Efficient Tensor Migration and Allocation on Heterogeneous Memory Systems for Deep Learning. HPCA (2021)
Connor Holmes, Minjia Zhang, Yuxiong He, Bo Wu
NxMTransformer: Semi-Structured Sparsification for Natural Language Understanding via ADMM. CoRR (2021)
Connor Holmes, Minjia Zhang, Yuxiong He, Bo Wu
NxMTransformer: Semi-Structured Sparsification for Natural Language Understanding via ADMM. NeurIPS (2021)
Conglong Li, Minjia Zhang, David G. Andersen, Yuxiong He
Improving Approximate Nearest Neighbor Search through Learned Adaptive Early Termination. SIGMOD Conference (2020)
Minjia Zhang, Yuxiong He
Accelerating Training of Transformer-Based Language Models with Progressive Layer Dropping. CoRR (2020)
Minjia Zhang, Yuxiong He
Accelerating Training of Transformer-Based Language Models with Progressive Layer Dropping. NeurIPS (2020)
Menghao Li, Minjia Zhang, Chi Wang, Mingqin Li
AdaTune: Adaptive Tensor Program Compilation Made Efficient. NeurIPS (2020)
Jie Ren, Minjia Zhang, Dong Li
HM-ANN: Efficient Billion-Point Nearest Neighbor Search on Heterogeneous Memory. NeurIPS (2020)
Jie Ren, Jiaolin Luo, Kai Wu, Minjia Zhang, Dong Li
Sentinel: Runtime Data Management on Heterogeneous Main MemorySystems for Deep Learning. CoRR (2019)
Minjia Zhang, Samyam Rajbhandari, Wenhan Wang, Elton Zheng, Olatunji Ruwase, Jeff Rasley, Jason Li, Junhua Wang, Yuxiong He
Accelerating Large Scale Deep Learning Inference through DeepCPU at Microsoft. OpML (2019)
Minjia Zhang, Yuxiong He
GRIP: Multi-Store Capacity-Optimized High-Performance Nearest Neighbor Search for Vector Search Engine. CIKM (2019)
Reza Yazdani, Olatunji Ruwase, Minjia Zhang, Yuxiong He, José-María Arnau, Antonio González
LSTM-Sharp: An Adaptable, Energy-Efficient Hardware Accelerator for Long Short-Term Memory. CoRR (2019)
Junfeng Zhao, Minjia Zhang, Hongji Yang
Code Refactoring from OpenMP to MapReduce Model for Big Data Processing. SmartWorld/SCALCOM/UIC/ATC/CBDCom/IOP/SCI (2019)
Wei Wen, Yuxiong He, Samyam Rajbhandari, Minjia Zhang, Wenhan Wang, Fang Liu, Bin Hu, Yiran Chen, Hai Li
Learning Intrinsic Sparse Structures within Long Short-Term Memory. ICLR (Poster) (2018)
Minjia Zhang, Samyam Rajbhandari, Wenhan Wang, Yuxiong He
DeepCPU: Serving RNN-based Deep Learning Models 10x Faster. USENIX Annual Technical Conference (2018)
Minjia Zhang, Yuxiong He
Zoom: SSD-based Vector Search for Optimizing Accuracy, Latency and Memory. CoRR (2018)
Minjia Zhang, Xiaodong Liu, Wenhan Wang, Jianfeng Gao, Yuxiong He
Navigating with Graph Representations for Fast and Scalable Decoding of Neural Language Models. CoRR (2018)
Minjia Zhang, Wenhan Wang, Xiaodong Liu, Jianfeng Gao, Yuxiong He
Navigating with Graph Representations for Fast and Scalable Decoding of Neural Language Models. NeurIPS (2018)
Junfeng Zhao, Minjia Zhang
Refactoring OpenMP Code Based on MapReduce Model. ISPA/IUCC/BDCloud/SocialCom/SustainCom (2018)
Swarnendu Biswas, Man Cao, Minjia Zhang, Michael D. Bond, Benjamin P. Wood
Lightweight data race detection for production runs. CC (2017)
Man Cao, Minjia Zhang, Aritra Sengupta, Swarnendu Biswas, Michael D. Bond
Hybridizing and Relaxing Dependence Tracking for Efficient Parallel Runtime Support. ACM Trans. Parallel Comput. 4 (2) (2017)
Minjia Zhang, Swarnendu Biswas, Michael D. Bond
POSTER: On the Problem of Consistency Exceptions in the Context of Strong Memory Models. PPOPP (2017)
Minjia Zhang, Swarnendu Biswas, Michael D. Bond
Avoiding consistency exceptions under strong memory models. ISMM (2017)
Man Cao, Minjia Zhang, Aritra Sengupta, Michael D. Bond
Drinking from both glasses: combining pessimistic and optimistic tracking of cross-thread dependences. PPOPP (2016)
Minjia Zhang, Swarnendu Biswas, Michael D. Bond
Relaxed dependence tracking for parallel runtime support. CC (2016)
Minjia Zhang, Jipeng Huang, Man Cao, Michael D. Bond
Low-overhead software transactional memory with progress guarantees and strong semantics. PPOPP (2015)
Aritra Sengupta, Swarnendu Biswas, Minjia Zhang, Michael D. Bond, Milind Kulkarni
Hybrid Static: Dynamic Analysis for Statically Bounded Region Serializability. ASPLOS (2015)
Minjia Zhang
SIRe: an efficient snapshot isolation-based memory model for detecting and tolerating region conflicts. SPLASH (Companion Volume) (2015)
Swarnendu Biswas, Minjia Zhang, Michael D. Bond, Brandon Lucia
Valor: efficient, software-only region conflict exceptions. OOPSLA (2015)
Michael D. Bond, Milind Kulkarni, Man Cao, Minjia Zhang, Meisam Fathi Salmi, Swarnendu Biswas, Aritra Sengupta, Jipeng Huang
OCTET: capturing and controlling cross-thread dependences efficiently. OOPSLA (2013)
Jithin Jose, Hari Subramoni, Miao Luo, Minjia Zhang, Jian Huang, Md. Wasi-ur-Rahman, Nusrat S. Islam, Xiangyong Ouyang, Hao Wang, Sayantan Sur, Dhabaleswar K. Panda
Memcached Design on High Performance RDMA Capable Interconnects. ICPP (2011)
Minjia Zhang, Hai Jin, Xuanhua Shi, Song Wu
VirtCFT: A Transparent VM-Level Fault-Tolerant System for Virtual Clusters. ICPADS (2010)