Zhewei Yao

Publication Activity (10 Years)

Years Active: 2017-2024
Publications (10 Years): 86

Top Topics

Intravascular Ultrasound

Top Venues

Publications

Haojun Xia, Zhen Zheng, Xiaoxia Wu, Shiyang Chen, Zhewei Yao, Stephen Youn, Arash Bakhtiari, Michael Wyatt, Donglin Zhuang, Zhongzhu Zhou, Olatunji Ruwase, Yuxiong He, Shuaiwen Leon Song
Quant-LLM: Accelerating the Serving of Large Language Models via FP6-Centric Algorithm-System Co-Design on Modern GPUs. USENIX ATC (2024)
Guanhua Wang, Heyang Qin, Sam Ade Jacobs, Xiaoxia Wu, Connor Holmes, Zhewei Yao, Samyam Rajbhandari, Olatunji Ruwase, Feng Yan, Lei Yang, Yuxiong He
ZeRO++: Extremely Efficient Collective Communication for Large Model Training. ICLR (2024)
Zhewei Yao, Xiaoxia Wu, Cheng Li, Stephen Youn, Yuxiong He
Exploring Post-training Quantization in LLMs from Comprehensive Study to Low Rank Compensation. AAAI (2024)
Amir Gholami, Zhewei Yao, Sehoon Kim, Coleman Hooper, Michael W. Mahoney, Kurt Keutzer
AI and Memory Wall. IEEE Micro 44 (3) (2024)
Amir Gholami, Zhewei Yao, Sehoon Kim, Coleman Hooper, Michael W. Mahoney, Kurt Keutzer
AI and Memory Wall. CoRR (2024)
Zhenyu Zhang, Runjin Chen, Shiwei Liu, Zhewei Yao, Olatunji Ruwase, Beidi Chen, Xiaoxia Wu, Zhangyang Wang
Found in the Middle: How Language Models Use Long Contexts Better via Plug-and-Play Positional Encoding. CoRR (2024)
Haojun Xia, Zhen Zheng, Xiaoxia Wu, Shiyang Chen, Zhewei Yao, Stephen Youn, Arash Bakhtiari, Michael Wyatt, Donglin Zhuang, Zhongzhu Zhou, Olatunji Ruwase, Yuxiong He, Shuaiwen Leon Song
FP6-LLM: Efficiently Serving Large Language Models Through FP6-Centric Algorithm-System Co-Design. CoRR (2024)
Conglong Li, Zhewei Yao, Xiaoxia Wu, Minjia Zhang, Connor Holmes, Cheng Li, Yuxiong He
DeepSpeed Data Efficiency: Improving Deep Learning Model Quality and Training Efficiency via Efficient Data Sampling and Routing. AAAI (2024)
Syed Zawad, Cheng Li, Zhewei Yao, Elton Zheng, Yuxiong He, Feng Yan
DySR: Adaptive Super-Resolution via Algorithm and System Co-design. ICLR (2023)
Sheng Shen, Zhewei Yao, Chunyuan Li, Trevor Darrell, Kurt Keutzer, Yuxiong He
Scaling Vision-Language Models with Sparse Mixture of Experts. EMNLP (Findings) (2023)
Fengxiang Bie, Yibo Yang, Zhongzhu Zhou, Adam Ghanem, Minjia Zhang, Zhewei Yao, Xiaoxia Wu, Connor Holmes, Pareesa Ameneh Golnari, David A. Clifton, Yuxiong He, Dacheng Tao, Shuaiwen Leon Song
RenAIssance: A Survey into AI Text-to-Image Generation in the Era of Large Model. CoRR (2023)
Zhewei Yao, Xiaoxia Wu, Conglong Li, Minjia Zhang, Heyang Qin, Olatunji Ruwase, Ammar Ahmad Awan, Samyam Rajbhandari, Yuxiong He
DeepSpeed-VisualChat: Multi-Round Multi-Image Interleave Chat via Multi-Modal Causal Attention. CoRR (2023)
Zhewei Yao, Reza Yazdani Aminabadi, Stephen Youn, Xiaoxia Wu, Elton Zheng, Yuxiong He
ZeroQuant-HERO: Hardware-Enhanced Robust Optimized Post-Training Quantization Framework for W8A8 Transformers. CoRR (2023)
Zhewei Yao, Reza Yazdani Aminabadi, Olatunji Ruwase, Samyam Rajbhandari, Xiaoxia Wu, Ammar Ahmad Awan, Jeff Rasley, Minjia Zhang, Conglong Li, Connor Holmes, Zhongzhu Zhou, Michael Wyatt, Molly Smith, Lev Kurilenko, Heyang Qin, Masahiro Tanaka, Shuai Che, Shuaiwen Leon Song, Yuxiong He
DeepSpeed-Chat: Easy, Fast and Affordable RLHF Training of ChatGPT-like Models at All Scales. CoRR (2023)
Xiaoxia Wu, Zhewei Yao, Yuxiong He
ZeroQuant-FP: A Leap Forward in LLMs Post-Training W4A8 Quantization Using Floating-Point Formats. CoRR (2023)
Xiaoxia Wu, Cheng Li, Reza Yazdani Aminabadi, Zhewei Yao, Yuxiong He
Understanding Int4 Quantization for Language Models: Latency Speedup, Composability, and Failure Cases. ICML (2023)
Sheng Shen, Zhewei Yao, Chunyuan Li, Trevor Darrell, Kurt Keutzer, Yuxiong He
Scaling Vision-Language Models with Sparse Mixture of Experts. CoRR (2023)
Zhewei Yao, Cheng Li, Xiaoxia Wu, Stephen Youn, Yuxiong He
A Comprehensive Study on Post-Training Quantization for Large Language Models. CoRR (2023)
Pareesa Ameneh Golnari, Zhewei Yao, Yuxiong He
Selective Guidance: Are All the Denoising Steps of Guided Diffusion Important? CoRR (2023)
Xiaoxia Wu, Cheng Li, Reza Yazdani Aminabadi, Zhewei Yao, Yuxiong He
Understanding INT4 Quantization for Transformer Models: Latency Speedup, Composability, and Failure Cases. CoRR (2023)
Xiaoxia Wu, Haojun Xia, Stephen Youn, Zhen Zheng, Shiyang Chen, Arash Bakhtiari, Michael Wyatt, Reza Yazdani Aminabadi, Yuxiong He, Olatunji Ruwase, Leon Song, Zhewei Yao
ZeroQuant(4+2): Redefining LLMs Quantization with a New FP6-Centric Strategy for Diverse Generative Tasks. CoRR (2023)
Xiaoxia Wu, Zhewei Yao, Minjia Zhang, Conglong Li, Yuxiong He
Extreme Compression for Pre-trained Transformers Made Simple and Efficient. CoRR (2022)
Samyam Rajbhandari, Conglong Li, Zhewei Yao, Minjia Zhang, Reza Yazdani Aminabadi, Ammar Ahmad Awan, Jeff Rasley, Yuxiong He
DeepSpeed-MoE: Advancing Mixture-of-Experts Inference and Training to Power Next-Generation AI Scale. ICML (2022)
Shixing Yu, Zhewei Yao, Amir Gholami, Zhen Dong, Sehoon Kim, Michael W. Mahoney, Kurt Keutzer
Hessian-Aware Pruning and Optimal Neural Implant. WACV (2022)
Yuxin Ma, Ping Gong, Jun Yi, Zhewei Yao, Minjie Wang, Cheng Li, Yuxiong He, Feng Yan
BiFeat: Supercharge GNN Training via Graph Feature Quantization. CoRR (2022)
Sehoon Kim, Amir Gholami, Zhewei Yao, Nicholas Lee, Patrick Wang, Aniruddha Nrusimha, Bohan Zhai, Tianren Gao, Michael W. Mahoney, Kurt Keutzer
Integer-Only Zero-Shot Quantization for Efficient Speech Recognition. ICASSP (2022)
Zhewei Yao, Xiaoxia Wu, Conglong Li, Connor Holmes, Minjia Zhang, Cheng Li, Yuxiong He
Random-LTD: Random and Layerwise Token Dropping Brings Efficient Training for Large-scale Transformers. CoRR (2022)
Sheng Shen, Liunian Harold Li, Hao Tan, Mohit Bansal, Anna Rohrbach, Kai-Wei Chang, Zhewei Yao, Kurt Keutzer
How Much Can CLIP Benefit Vision-and-Language Tasks? ICLR (2022)
Zhewei Yao, Reza Yazdani Aminabadi, Minjia Zhang, Xiaoxia Wu, Conglong Li, Yuxiong He
ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformers. CoRR (2022)
Conglong Li, Zhewei Yao, Xiaoxia Wu, Minjia Zhang, Yuxiong He
DeepSpeed Data Efficiency: Improving Deep Learning Model Quality and Training Efficiency via Efficient Data Sampling and Routing. CoRR (2022)
Xiaoxia Wu, Zhewei Yao, Minjia Zhang, Conglong Li, Yuxiong He
XTC: Extreme Compression for Pre-trained Transformers Made Simple and Efficient. NeurIPS (2022)
Samyam Rajbhandari, Conglong Li, Zhewei Yao, Minjia Zhang, Reza Yazdani Aminabadi, Ammar Ahmad Awan, Jeff Rasley, Yuxiong He
DeepSpeed-MoE: Advancing Mixture-of-Experts Inference and Training to Power Next-Generation AI Scale. CoRR (2022)
Zhewei Yao, Reza Yazdani Aminabadi, Minjia Zhang, Xiaoxia Wu, Conglong Li, Yuxiong He
ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformers. NeurIPS (2022)
Sheng Shen, Zhewei Yao, Douwe Kiela, Kurt Keutzer, Michael W. Mahoney
What's Hidden in a One-layer Randomly Weighted Transformer? EMNLP (1) (2021)
Jianfei Chen, Lianmin Zheng, Zhewei Yao, Dequan Wang, Ion Stoica, Michael W. Mahoney, Joseph E. Gonzalez
ActNN: Reducing Training Memory Footprint via 2-Bit Activation Compressed Training. CoRR (2021)
Zhewei Yao, Zhen Dong, Zhangcheng Zheng, Amir Gholami, Jiali Yu, Eric Tan, Leyuan Wang, Qijing Huang, Yida Wang, Michael W. Mahoney, Kurt Keutzer
HAWQ-V3: Dyadic Neural Network Quantization. ICML (2021)
Zhewei Yao, Peng Xu, Fred Roosta, Michael W. Mahoney
Inexact Nonconvex Newton-Type Methods. INFORMS J. Optim. 3 (2) (2021)
Sehoon Kim, Amir Gholami, Zhewei Yao, Aniruddha Nrusimha, Bohan Zhai, Tianren Gao, Michael W. Mahoney, Kurt Keutzer
Q-ASR: Integer-only Zero-shot Quantization for Efficient Speech Recognition. CoRR (2021)
Zhengming Zhang, Yaoqing Yang, Zhewei Yao, Yujun Yan, Joseph E. Gonzalez, Kannan Ramchandran, Michael W. Mahoney
Improving Semi-supervised Federated Learning by Reducing the Gradient Diversity of Models. IEEE BigData (2021)
Amir Gholami, Sehoon Kim, Zhen Dong, Zhewei Yao, Michael W. Mahoney, Kurt Keutzer
A Survey of Quantization Methods for Efficient Neural Network Inference. CoRR (2021)
Sehoon Kim, Amir Gholami, Zhewei Yao, Michael W. Mahoney, Kurt Keutzer
I-BERT: Integer-only BERT Quantization. ICML (2021)
Shixing Yu, Zhewei Yao, Amir Gholami, Zhen Dong, Michael W. Mahoney, Kurt Keutzer
Hessian-Aware Pruning and Optimal Neural Implant. CoRR (2021)
Sheng Shen, Zhewei Yao, Douwe Kiela, Kurt Keutzer, Michael W. Mahoney
What's Hidden in a One-layer Randomly Weighted Transformer? CoRR (2021)
Jianfei Chen, Lianmin Zheng, Zhewei Yao, Dequan Wang, Ion Stoica, Michael W. Mahoney, Joseph Gonzalez
ActNN: Reducing Training Memory Footprint via 2-Bit Activation Compressed Training. ICML (2021)
Sehoon Kim, Amir Gholami, Zhewei Yao, Michael W. Mahoney, Kurt Keutzer
I-BERT: Integer-only BERT Quantization. CoRR (2021)
Sheng Shen, Liunian Harold Li, Hao Tan, Mohit Bansal, Anna Rohrbach, Kai-Wei Chang, Zhewei Yao, Kurt Keutzer
How Much Can CLIP Benefit Vision-and-Language Tasks? CoRR (2021)
Zhewei Yao, Linjian Ma, Sheng Shen, Kurt Keutzer, Michael W. Mahoney
MLPruning: A Multilevel Structured Pruning Framework for Transformer-based Models. CoRR (2021)
Zhewei Yao, Amir Gholami, Sheng Shen, Mustafa Mustafa, Kurt Keutzer, Michael W. Mahoney
ADAHESSIAN: An Adaptive Second Order Optimizer for Machine Learning. AAAI (2021)
Yaohui Cai, Zhewei Yao, Zhen Dong, Amir Gholami, Michael W. Mahoney, Kurt Keutzer
ZeroQ: A Novel Zero Shot Quantization Framework. CVPR (2020)
Zhen Dong, Zhewei Yao, Daiyaan Arfeen, Amir Gholami, Michael W. Mahoney, Kurt Keutzer
HAWQ-V2: Hessian Aware trace-Weighted Quantization of Neural Networks. NeurIPS (2020)
Qinxin Wang, Hao Tan, Sheng Shen, Michael W. Mahoney, Zhewei Yao
MAF: Multimodal Alignment Framework for Weakly-Supervised Phrase Grounding. EMNLP (1) (2020)
Linjian Ma, Gabe Montague, Jiayu Ye, Zhewei Yao, Amir Gholami, Kurt Keutzer, Michael W. Mahoney
Inefficiency of K-FAC for Large Batch Size Training. AAAI (2020)
Qinxin Wang, Hao Tan, Sheng Shen, Michael W. Mahoney, Zhewei Yao
MAF: Multimodal Alignment Framework for Weakly-Supervised Phrase Grounding. CoRR (2020)
Zhewei Yao, Amir Gholami, Kurt Keutzer, Michael W. Mahoney
PyHessian: Neural Networks Through the Lens of the Hessian. IEEE BigData (2020)
Zhewei Yao, Zhen Dong, Zhangcheng Zheng, Amir Gholami, Jiali Yu, Eric Tan, Leyuan Wang, Qijing Huang, Yida Wang, Michael W. Mahoney, Kurt Keutzer
HAWQV3: Dyadic Neural Network Quantization. CoRR (2020)
Yaohui Cai, Zhewei Yao, Zhen Dong, Amir Gholami, Michael W. Mahoney, Kurt Keutzer
ZeroQ: A Novel Zero Shot Quantization Framework. CoRR (2020)
Jianfei Chen, Yu Gai, Zhewei Yao, Michael W. Mahoney, Joseph E. Gonzalez
A Statistical Framework for Low-bitwidth Training of Deep Neural Networks. NeurIPS (2020)
Sheng Shen, Zhewei Yao, Amir Gholami, Michael W. Mahoney, Kurt Keutzer
PowerNorm: Rethinking Batch Normalization in Transformers. ICML (2020)
Sheng Shen, Zhen Dong, Jiayu Ye, Linjian Ma, Zhewei Yao, Amir Gholami, Michael W. Mahoney, Kurt Keutzer
Q-BERT: Hessian Based Ultra Low Precision Quantization of BERT. AAAI (2020)
Zhewei Yao, Amir Gholami, Sheng Shen, Kurt Keutzer, Michael W. Mahoney
ADAHESSIAN: An Adaptive Second Order Optimizer for Machine Learning. CoRR (2020)
Sheng Shen, Zhewei Yao, Amir Gholami, Michael W. Mahoney, Kurt Keutzer
Rethinking Batch Normalization in Transformers. CoRR (2020)
N. Benjamin Erichson, Zhewei Yao, Michael W. Mahoney
JumpReLU: A Retrofit Defense Strategy for Adversarial Attacks. ICPRAM (2020)
Zhengming Zhang, Zhewei Yao, Yaoqing Yang, Yujun Yan, Joseph E. Gonzalez, Michael W. Mahoney
Benchmarking Semi-supervised Federated Learning. CoRR (2020)
Jianfei Chen, Yu Gai, Zhewei Yao, Michael W. Mahoney, Joseph E. Gonzalez
A Statistical Framework for Low-bitwidth Training of Deep Neural Networks. CoRR (2020)
Tianjun Zhang, Zhewei Yao, Amir Gholami, Kurt Keutzer, Joseph Gonzalez, George Biros, Michael W. Mahoney
ANODEV2: A Coupled Neural ODE Evolution Framework. CoRR (2019)
Zhewei Yao, Feng Yang, Jing Huang, Yaqin Liu
改进型循环生成对抗网络的血管内超声图像增强 (Improved CycleGANs for Intravascular Ultrasound Image Enhancement). 计算机科学 46 (5) (2019)
Zhen Dong, Zhewei Yao, Amir Gholami, Michael W. Mahoney, Kurt Keutzer
HAWQ: Hessian AWare Quantization of Neural Networks With Mixed-Precision. ICCV (2019)
N. Benjamin Erichson, Lionel Mathelin, Zhewei Yao, Steven L. Brunton, Michael W. Mahoney, J. Nathan Kutz
Shallow Learning for Fluid Flow Reconstruction with Limited Sensors and Limited Data. CoRR (2019)
N. Benjamin Erichson, Zhewei Yao, Michael W. Mahoney
JumpReLU: A Retrofit Defense Strategy for Adversarial Attacks. CoRR (2019)
Linjian Ma, Gabe Montague, Jiayu Ye, Zhewei Yao, Amir Gholami, Kurt Keutzer, Michael W. Mahoney
Inefficiency of K-FAC for Large Batch Size Training. CoRR (2019)
Sheng Shen, Zhen Dong, Jiayu Ye, Linjian Ma, Zhewei Yao, Amir Gholami, Michael W. Mahoney, Kurt Keutzer
Q-BERT: Hessian Based Ultra Low Precision Quantization of BERT. CoRR (2019)
Zhewei Yao, Amir Gholami, Peng Xu, Kurt Keutzer, Michael W. Mahoney
Trust Region Based Adversarial Attack on Neural Networks. CVPR (2019)
Zhen Dong, Zhewei Yao, Yaohui Cai, Daiyaan Arfeen, Amir Gholami, Michael W. Mahoney, Kurt Keutzer
HAWQ-V2: Hessian Aware trace-Weighted Quantization of Neural Networks. CoRR (2019)
Kai Rothauge, Zhewei Yao, Zixi Hu, Michael W. Mahoney
Residual Networks as Nonlinear Systems: Stability Analysis using Linearization. CoRR (2019)
Tianjun Zhang, Zhewei Yao, Amir Gholami, Joseph E. Gonzalez, Kurt Keutzer, Michael W. Mahoney, George Biros
ANODEV2: A Coupled Neural ODE Framework. NeurIPS (2019)
Zhen Dong, Zhewei Yao, Amir Gholami, Michael W. Mahoney, Kurt Keutzer
HAWQ: Hessian AWare Quantization of Neural Networks with Mixed-Precision. CoRR (2019)
Zhewei Yao, Amir Gholami, Kurt Keutzer, Michael W. Mahoney
PyHessian: Neural Networks Through the Lens of the Hessian. CoRR (2019)
Zhewei Yao, Amir Gholami, Kurt Keutzer, Michael W. Mahoney
Large batch size training of neural networks with adversarial training and second-order information. CoRR (2018)
Zhewei Yao, Amir Gholami, Peng Xu, Kurt Keutzer, Michael W. Mahoney
Trust Region Based Adversarial Attack on Neural Networks. CoRR (2018)
Zhewei Yao, Amir Gholami, Qi Lei, Kurt Keutzer, Michael W. Mahoney
Hessian-based Analysis of Large Batch Training and Robustness to Adversaries. CoRR (2018)
Noah Golmant, Nikita Vemuri, Zhewei Yao, Vladimir Feinberg, Amir Gholami, Kai Rothauge, Michael W. Mahoney, Joseph Gonzalez
On the Computational Inefficiency of Large Batch Sizes for Stochastic Gradient Descent. CoRR (2018)
Zhewei Yao, Amir Gholami, Qi Lei, Kurt Keutzer, Michael W. Mahoney
Hessian-based Analysis of Large Batch Training and Robustness to Adversaries. NeurIPS (2018)
Norman Mu, Zhewei Yao, Amir Gholami, Kurt Keutzer, Michael W. Mahoney
Parameter Re-Initialization through Cyclical Batch Size Schedules. CoRR (2018)
Qingping Zhou, Zixi Hu, Zhewei Yao, Jinglai Li
A Hybrid Adaptive MCMC Algorithm in Function Spaces. SIAM/ASA J. Uncertain. Quantification 5 (1) (2017)
Zixi Hu, Zhewei Yao, Jinglai Li
On an adaptive preconditioned Crank-Nicolson MCMC algorithm for infinite dimensional Bayesian inference. J. Comput. Phys. 332 (2017)
Shujun Liang, Feng Yang, Tiexiang Wen, Zhewei Yao, Qinghua Huang, Chengke Ye
Nonlocal total variation based on symmetric Kullback-Leibler divergence for the ultrasound image despeckling. BMC Medical Imaging 17 (1) (2017)