Li Dong

ORCID

Publication Activity (10 Years)

Years Active: 2011-2024
Publications (10 Years): 153

Top Venues

Publications

Yuxian Gu, Li Dong, Yaru Hao, Qingxiu Dong, Minlie Huang, Furu Wei
Towards Optimal Learning of Language Models. CoRR (2024)
Shuming Ma, Hongyu Wang, Lingxiao Ma, Lei Wang, Wenhui Wang, Shaohan Huang, Li Dong, Ruiping Wang, Jilong Xue, Furu Wei
The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits. CoRR (2024)
Xichen Pan, Li Dong, Shaohan Huang, Zhiliang Peng, Wenhu Chen, Furu Wei
Kosmos-G: Generating Images in Context with Multimodal Large Language Models. ICLR (2024)
Wenshan Wu, Shaoguang Mao, Yadong Zhang, Yan Xia, Li Dong, Lei Cui, Furu Wei
Visualization-of-Thought Elicits Spatial Reasoning in Large Language Models. CoRR (2024)
Hangbo Bao, Li Dong, Wenhui Wang, Nan Yang, Songhao Piao, Furu Wei
Fine-tuning pretrained transformer encoders for sequence-to-sequence learning. Int. J. Mach. Learn. Cybern. 15 (5) (2024)
Yuxian Gu, Li Dong, Furu Wei, Minlie Huang
MiniLLM: Knowledge Distillation of Large Language Models. ICLR (2024)
Zonglin Yang, Li Dong, Xinya Du, Hao Cheng, Erik Cambria, Xiaodong Liu, Jianfeng Gao, Furu Wei
Language Models as Inductive Reasoners. EACL (1) (2024)
Zhiliang Peng, Wenhui Wang, Li Dong, Yaru Hao, Shaohan Huang, Shuming Ma, Qixiang Ye, Furu Wei
Grounding Multimodal Large Language Models to the World. ICLR (2024)
Yutao Sun, Li Dong, Yi Zhu, Shaohan Huang, Wenhui Wang, Shuming Ma, Quanlu Zhang, Jianyong Wang, Furu Wei
You Only Cache Once: Decoder-Decoder Architectures for Language Models. CoRR (2024)
Yuxian Gu, Li Dong, Furu Wei, Minlie Huang
Pre-Training to Learn in Context. ACL (1) (2023)
Hongyu Wang, Shuming Ma, Shaohan Huang, Li Dong, Wenhui Wang, Zhiliang Peng, Yu Wu, Payal Bajaj, Saksham Singhal, Alon Benhaim, Barun Patra, Zhun Liu, Vishrav Chaudhary, Xia Song, Furu Wei
Magneto: A Foundation Transformer. ICML (2023)
Zhiliang Peng, Wenhui Wang, Li Dong, Yaru Hao, Shaohan Huang, Shuming Ma, Furu Wei
Kosmos-2: Grounding Multimodal Large Language Models to the World. CoRR (2023)
Hongyu Wang, Shuming Ma, Li Dong, Shaohan Huang, Huaijie Wang, Lingxiao Ma, Fan Yang, Ruiping Wang, Yi Wu, Furu Wei
BitNet: Scaling 1-bit Transformers for Large Language Models. CoRR (2023)
Jian Yang, Shuming Ma, Li Dong, Shaohan Huang, Haoyang Huang, Yuwei Yin, Dongdong Zhang, Liqun Yang, Furu Wei, Zhoujun Li
GanLM: Encoder-Decoder Pre-training with an Auxiliary Discriminator. ACL (1) (2023)
Zhiliang Peng, Li Dong, Hangbo Bao, Furu Wei, Qixiang Ye
A Unified View of Masked Image Modeling. Trans. Mach. Learn. Res. 2023 (2023)
Damai Dai, Yutao Sun, Li Dong, Yaru Hao, Shuming Ma, Zhifang Sui, Furu Wei
Why Can GPT Learn In-Context? Language Models Secretly Perform Gradient Descent as Meta-Optimizers. ACL (Findings) (2023)
Xichen Pan, Li Dong, Shaohan Huang, Zhiliang Peng, Wenhu Chen, Furu Wei
Kosmos-G: Generating Images in Context with Multimodal Large Language Models. CoRR (2023)
Yuxin Fang, Li Dong, Hangbo Bao, Xinggang Wang, Furu Wei
Corrupted Image Modeling for Self-Supervised Visual Pre-Training. ICLR (2023)
Yuxian Gu, Li Dong, Furu Wei, Minlie Huang
Pre-Training to Learn in Context. CoRR (2023)
Weizhi Wang, Li Dong, Hao Cheng, Xiaodong Liu, Xifeng Yan, Jianfeng Gao, Furu Wei
Augmenting Language Models with Long-Term Memory. NeurIPS (2023)
Weizhi Wang, Li Dong, Hao Cheng, Xiaodong Liu, Xifeng Yan, Jianfeng Gao, Furu Wei
Augmenting Language Models with Long-Term Memory. CoRR (2023)
Qingxiu Dong, Li Dong, Ke Xu, Guangyan Zhou, Yaru Hao, Zhifang Sui, Furu Wei
Large Language Model for Science: A Study on P vs. NP. CoRR (2023)
Tao Ge, Hu Jing, Li Dong, Shaoguang Mao, Yan Xia, Xun Wang, Si-Qing Chen, Furu Wei
Extensible Prompts for Language Models on Zero-shot Language Style Customization. NeurIPS (2023)
Jiayu Ding, Shuming Ma, Li Dong, Xingxing Zhang, Shaohan Huang, Wenhui Wang, Nanning Zheng, Furu Wei
LongNet: Scaling Transformers to 1, 000, 000, 000 Tokens. CoRR (2023)
Yaru Hao, Zewen Chi, Li Dong, Furu Wei
Optimizing Prompts for Text-to-Image Generation. NeurIPS (2023)
Zhixiong Han, Yaru Hao, Li Dong, Yutao Sun, Furu Wei
Prototypical Calibration for Few-shot Learning of Language Models. ICLR (2023)
Shaohan Huang, Li Dong, Wenhui Wang, Yaru Hao, Saksham Singhal, Shuming Ma, Tengchao Lv, Lei Cui, Owais Khan Mohammed, Barun Patra, Qiang Liu, Kriti Aggarwal, Zewen Chi, Nils Johan Bertil Bjorck, Vishrav Chaudhary, Subhojit Som, Xia Song, Furu Wei
Language Is Not All You Need: Aligning Perception with Language Models. NeurIPS (2023)
Wei Huang, Zhiliang Peng, Li Dong, Furu Wei, Jianbin Jiao, Qixiang Ye
Generic-to-Specific Distillation of Masked Autoencoders. CoRR (2023)
Tengchao Lv, Yupan Huang, Jingye Chen, Lei Cui, Shuming Ma, Yaoyao Chang, Shaohan Huang, Wenhui Wang, Li Dong, Weiyao Luo, Shaoxiang Wu, Guoxin Wang, Cha Zhang, Furu Wei
Kosmos-2.5: A Multimodal Literate Model. CoRR (2023)
Yuxian Gu, Li Dong, Furu Wei, Minlie Huang
Knowledge Distillation of Large Language Models. CoRR (2023)
Weizhi Wang, Li Dong, Hao Cheng, Haoyu Song, Xiaodong Liu, Xifeng Yan, Jianfeng Gao, Furu Wei
Visually-Augmented Language Modeling. ICLR (2023)
Yutao Sun, Li Dong, Shaohan Huang, Shuming Ma, Yuqing Xia, Jilong Xue, Jianyong Wang, Furu Wei
Retentive Network: A Successor to Transformer for Large Language Models. CoRR (2023)
Shaohan Huang, Li Dong, Wenhui Wang, Yaru Hao, Saksham Singhal, Shuming Ma, Tengchao Lv, Lei Cui, Owais Khan Mohammed, Qiang Liu, Kriti Aggarwal, Zewen Chi, Johan Bjorck, Vishrav Chaudhary, Subhojit Som, Xia Song, Furu Wei
Language Is Not All You Need: Aligning Perception with Language Models. CoRR (2023)
Wenhui Wang, Hangbo Bao, Li Dong, Johan Bjorck, Zhiliang Peng, Qiang Liu, Kriti Aggarwal, Owais Khan Mohammed, Saksham Singhal, Subhojit Som, Furu Wei
Image as a Foreign Language: BEIT Pretraining for Vision and Vision-Language Tasks. CVPR (2023)
Jinghao Zhou, Li Dong, Zhe Gan, Lijuan Wang, Furu Wei
Non-Contrastive Learning Meets Language-Image Pre-Training. CVPR (2023)
Wei Huang, Zhiliang Peng, Li Dong, Furu Wei, Jianbin Jiao, Qixiang Ye
Generic-to-Specific Distillation of Masked Autoencoders. CVPR (2023)
Barun Patra, Saksham Singhal, Shaohan Huang, Zewen Chi, Li Dong, Furu Wei, Vishrav Chaudhary, Xia Song
Beyond English-Centric Bitexts for Better Multilingual Language Representation Learning. ACL (1) (2023)
Yutao Sun, Li Dong, Barun Patra, Shuming Ma, Shaohan Huang, Alon Benhaim, Vishrav Chaudhary, Xia Song, Furu Wei
A Length-Extrapolatable Transformer. ACL (1) (2023)
Haoyu Song, Li Dong, Wei-Nan Zhang, Ting Liu, Furu Wei
CLIP Models are Few-shot Learners: Empirical Studies on VQA and Visual Entailment. CoRR (2022)
Jian Yang, Shaohan Huang, Shuming Ma, Yuwei Yin, Li Dong, Dongdong Zhang, Hongcheng Guo, Zhoujun Li, Furu Wei
CROP: Zero-shot Cross-lingual Named Entity Recognition with Multilingual Labeled Sequence Translation. CoRR (2022)
Jinghao Zhou, Li Dong, Zhe Gan, Lijuan Wang, Furu Wei
Non-Contrastive Learning Meets Language-Image Pre-Training. CoRR (2022)
Shuming Ma, Hongyu Wang, Shaohan Huang, Wenhui Wang, Zewen Chi, Li Dong, Alon Benhaim, Barun Patra, Vishrav Chaudhary, Xia Song, Furu Wei
TorchScale: Transformers at Scale. CoRR (2022)
Zewen Chi, Shaohan Huang, Li Dong, Shuming Ma, Bo Zheng, Saksham Singhal, Payal Bajaj, Xia Song, Xian-Ling Mao, Heyan Huang, Furu Wei
XLM-E: Cross-lingual Language Model Pre-training via ELECTRA. ACL (1) (2022)
Barun Patra, Saksham Singhal, Shaohan Huang, Zewen Chi, Li Dong, Furu Wei, Vishrav Chaudhary, Xia Song
Beyond English-Centric Bitexts for Better Multilingual Language Representation Learning. CoRR (2022)
Jing Qian, Li Dong, Yelong Shen, Furu Wei, Weizhu Chen
Controllable Natural Language Generation with Contrastive Prefixes. ACL (Findings) (2022)
Tianyu Chen, Hangbo Bao, Shaohan Huang, Li Dong, Binxing Jiao, Daxin Jiang, Haoyi Zhou, Jianxin Li, Furu Wei
THE-X: Privacy-Preserving Transformer Inference with Homomorphic Encryption. CoRR (2022)
Damai Dai, Li Dong, Shuming Ma, Bo Zheng, Zhifang Sui, Baobao Chang, Furu Wei
StableMoE: Stable Routing Strategy for Mixture of Experts. ACL (1) (2022)
Zhixiong Han, Yaru Hao, Li Dong, Furu Wei
Prototypical Calibration for Few-shot Learning of Language Models. CoRR (2022)
Weizhi Wang, Li Dong, Hao Cheng, Haoyu Song, Xiaodong Liu, Xifeng Yan, Jianfeng Gao, Furu Wei
Visually-Augmented Language Modeling. CoRR (2022)
Haoyu Song, Li Dong, Weinan Zhang, Ting Liu, Furu Wei
CLIP Models are Few-Shot Learners: Empirical Studies on VQA and Visual Entailment. ACL (1) (2022)
Da Yin, Li Dong, Hao Cheng, Xiaodong Liu, Kai-Wei Chang, Furu Wei, Jianfeng Gao
A Survey of Knowledge-Intensive NLP with Pre-Trained Language Models. CoRR (2022)
Yaru Hao, Haoyu Song, Li Dong, Shaohan Huang, Zewen Chi, Wenhui Wang, Shuming Ma, Furu Wei
Language Models are General-Purpose Interfaces. CoRR (2022)
Damai Dai, Li Dong, Yaru Hao, Zhifang Sui, Baobao Chang, Furu Wei
Knowledge Neurons in Pretrained Transformers. ACL (1) (2022)
Jian Yang, Shaohan Huang, Shuming Ma, Yuwei Yin, Li Dong, Dongdong Zhang, Hongcheng Guo, Zhoujun Li, Furu Wei
CROP: Zero-shot Cross-lingual Named Entity Recognition with Multilingual Labeled Sequence Translation. EMNLP (Findings) (2022)
Yunzhi Yao, Shaohan Huang, Ningyu Zhang, Li Dong, Furu Wei, Huajun Chen
Kformer: Knowledge Injection in Transformer Feed-Forward Layers. CoRR (2022)
Zhiliang Peng, Li Dong, Hangbo Bao, Qixiang Ye, Furu Wei
A Unified View of Masked Image Modeling. CoRR (2022)
Jian Yang, Shuming Ma, Li Dong, Shaohan Huang, Haoyang Huang, Yuwei Yin, Dongdong Zhang, Liqun Yang, Zhoujun Li, Furu Wei
GanLM: Encoder-Decoder Pre-training with an Auxiliary Discriminator. CoRR (2022)
Zewen Chi, Li Dong, Shaohan Huang, Damai Dai, Shuming Ma, Barun Patra, Saksham Singhal, Payal Bajaj, Xia Song, Xian-Ling Mao, Heyan Huang, Furu Wei
On the Representation Collapse of Sparse Mixture of Experts. NeurIPS (2022)
Yunzhi Yao, Shaohan Huang, Li Dong, Furu Wei, Huajun Chen, Ningyu Zhang
Kformer: Knowledge Injection in Transformer Feed-Forward Layers. NLPCC (1) (2022)
Damai Dai, Li Dong, Shuming Ma, Bo Zheng, Zhifang Sui, Baobao Chang, Furu Wei
StableMoE: Stable Routing Strategy for Mixture of Experts. CoRR (2022)
Yuxin Fang, Li Dong, Hangbo Bao, Xinggang Wang, Furu Wei
Corrupted Image Modeling for Self-Supervised Visual Pre-Training. CoRR (2022)
Haichao Zhu, Li Dong, Furu Wei, Bing Qin, Ting Liu
Transforming Wikipedia Into Augmented Data for Query-Focused Summarization. IEEE ACM Trans. Audio Speech Lang. Process. 30 (2022)
Jing Qian, Li Dong, Yelong Shen, Furu Wei, Weizhu Chen
Controllable Natural Language Generation with Contrastive Prefixes. CoRR (2022)
Wenhui Wang, Hangbo Bao, Li Dong, Johan Bjorck, Zhiliang Peng, Qiang Liu, Kriti Aggarwal, Owais Khan Mohammed, Saksham Singhal, Subhojit Som, Furu Wei
Image as a Foreign Language: BEiT Pretraining for All Vision and Vision-Language Tasks. CoRR (2022)
Ze Liu, Han Hu, Yutong Lin, Zhuliang Yao, Zhenda Xie, Yixuan Wei, Jia Ning, Yue Cao, Zheng Zhang, Li Dong, Furu Wei, Baining Guo
Swin Transformer V2: Scaling Up Capacity and Resolution. CVPR (2022)
Yutao Sun, Li Dong, Barun Patra, Shuming Ma, Shaohan Huang, Alon Benhaim, Vishrav Chaudhary, Xia Song, Furu Wei
A Length-Extrapolatable Transformer. CoRR (2022)
Hongyu Wang, Shuming Ma, Li Dong, Shaohan Huang, Dongdong Zhang, Furu Wei
DeepNet: Scaling Transformers to 1, 000 Layers. CoRR (2022)
Tianyu Chen, Hangbo Bao, Shaohan Huang, Li Dong, Binxing Jiao, Daxin Jiang, Haoyi Zhou, Jianxin Li, Furu Wei
THE-X: Privacy-Preserving Transformer Inference with Homomorphic Encryption. ACL (Findings) (2022)
Hangbo Bao, Wenhui Wang, Li Dong, Furu Wei
VL-BEiT: Generative Vision-Language Pretraining. CoRR (2022)
Yaru Hao, Yutao Sun, Li Dong, Zhixiong Han, Yuxian Gu, Furu Wei
Structured Prompting: Scaling In-Context Learning to 1, 000 Examples. CoRR (2022)
Hangbo Bao, Wenhui Wang, Li Dong, Qiang Liu, Owais Khan Mohammed, Kriti Aggarwal, Subhojit Som, Songhao Piao, Furu Wei
VLMo: Unified Vision-Language Pre-Training with Mixture-of-Modality-Experts. NeurIPS (2022)
Yaru Hao, Zewen Chi, Li Dong, Furu Wei
Optimizing Prompts for Text-to-Image Generation. CoRR (2022)
Tao Ge, Jing Hu, Li Dong, Shaoguang Mao, Yan Xia, Xun Wang, Si-Qing Chen, Furu Wei
Extensible Prompts for Language Models. CoRR (2022)
Damai Dai, Yutao Sun, Li Dong, Yaru Hao, Zhifang Sui, Furu Wei
Why Can GPT Learn In-Context? Language Models Secretly Perform Gradient Descent as Meta-Optimizers. CoRR (2022)
Hongyu Wang, Shuming Ma, Shaohan Huang, Li Dong, Wenhui Wang, Zhiliang Peng, Yu Wu, Payal Bajaj, Saksham Singhal, Alon Benhaim, Barun Patra, Zhun Liu, Vishrav Chaudhary, Xia Song, Furu Wei
Foundation Transformers. CoRR (2022)
Hangbo Bao, Li Dong, Songhao Piao, Furu Wei
BEiT: BERT Pre-Training of Image Transformers. ICLR (2022)
Zhiliang Peng, Li Dong, Hangbo Bao, Qixiang Ye, Furu Wei
BEiT v2: Masked Image Modeling with Vector-Quantized Visual Tokenizers. CoRR (2022)
Zewen Chi, Li Dong, Shaohan Huang, Damai Dai, Shuming Ma, Barun Patra, Saksham Singhal, Payal Bajaj, Xia Song, Furu Wei
On the Representation Collapse of Sparse Mixture of Experts. CoRR (2022)
Ze Liu, Han Hu, Yutong Lin, Zhuliang Yao, Zhenda Xie, Yixuan Wei, Jia Ning, Yue Cao, Zheng Zhang, Li Dong, Furu Wei, Baining Guo
Swin Transformer V2: Scaling Up Capacity and Resolution. CoRR (2021)
Zewen Chi, Shaohan Huang, Li Dong, Shuming Ma, Saksham Singhal, Payal Bajaj, Xia Song, Furu Wei
XLM-E: Cross-lingual Language Model Pre-training via ELECTRA. CoRR (2021)
Shuming Ma, Li Dong, Shaohan Huang, Dongdong Zhang, Alexandre Muzio, Saksham Singhal, Hany Hassan Awadalla, Xia Song, Furu Wei
DeltaLM: Encoder-Decoder Pre-training for Language Generation and Translation by Augmenting Pretrained Multilingual Encoders. CoRR (2021)
Hangbo Bao, Li Dong, Wenhui Wang, Nan Yang, Furu Wei
s2s-ft: Fine-Tuning Pretrained Transformer Encoders for Sequence-to-Sequence Learning. CoRR (2021)
Wenhui Wang, Hangbo Bao, Li Dong, Furu Wei
VLMo: Unified Vision-Language Pre-Training with Mixture-of-Modality-Experts. CoRR (2021)
Bo Zheng, Li Dong, Shaohan Huang, Wenhui Wang, Zewen Chi, Saksham Singhal, Wanxiang Che, Ting Liu, Xia Song, Furu Wei
Consistency Regularization for Cross-Lingual Fine-Tuning. CoRR (2021)
Yuekai Zhao, Li Dong, Yelong Shen, Zhihua Zhang, Furu Wei, Weizhu Chen
Memory-Efficient Differentiable Transformer Architecture Search. ACL/IJCNLP (Findings) (2021)
Wenhui Wang, Hangbo Bao, Shaohan Huang, Li Dong, Furu Wei
MiniLMv2: Multi-Head Self-Attention Relation Distillation for Compressing Pretrained Transformers. ACL/IJCNLP (Findings) (2021)
Yunzhi Yao, Shaohan Huang, Wenhui Wang, Li Dong, Furu Wei
Adapt-and-Distill: Developing Small, Fast and Effective Pretrained Language Models for Domains. CoRR (2021)
Guanhua Chen, Shuming Ma, Yun Chen, Li Dong, Dongdong Zhang, Jia Pan, Wenping Wang, Furu Wei
Zero-shot Cross-lingual Transfer of Neural Machine Translation with Multilingual Pretrained Encoders. CoRR (2021)
Yaru Hao, Li Dong, Hangbo Bao, Ke Xu, Furu Wei
Learning to Sample Replacements for ELECTRA Pre-Training. ACL/IJCNLP (Findings) (2021)
Guanhua Chen, Shuming Ma, Yun Chen, Li Dong, Dongdong Zhang, Jia Pan, Wenping Wang, Furu Wei
Zero-Shot Cross-Lingual Transfer of Neural Machine Translation with Multilingual Pretrained Encoders. EMNLP (1) (2021)
Hangbo Bao, Li Dong, Furu Wei
BEiT: BERT Pre-Training of Image Transformers. CoRR (2021)
Zewen Chi, Li Dong, Bo Zheng, Shaohan Huang, Xian-Ling Mao, Heyan Huang, Furu Wei
Improving Pretrained Cross-Lingual Language Models via Self-Labeled Word Alignment. ACL/IJCNLP (1) (2021)
Yunzhi Yao, Shaohan Huang, Wenhui Wang, Li Dong, Furu Wei
Adapt-and-Distill: Developing Small, Fast and Effective Pretrained Language Models for Domains. ACL/IJCNLP (Findings) (2021)
Yaru Hao, Li Dong, Hangbo Bao, Ke Xu, Furu Wei
Learning to Sample Replacements for ELECTRA Pre-Training. CoRR (2021)
Zewen Chi, Li Dong, Shuming Ma, Shaohan Huang, Saksham Singhal, Xian-Ling Mao, Heyan Huang, Xia Song, Furu Wei
mT6: Multilingual Pretrained Text-to-Text Transformer with Translation Pairs. EMNLP (1) (2021)
Jian Yang, Shuming Ma, Haoyang Huang, Dongdong Zhang, Li Dong, Shaohan Huang, Alexandre Muzio, Saksham Singhal, Hany Hassan, Xia Song, Furu Wei
Multilingual Machine Translation Systems from Microsoft for WMT21 Shared Task. WMT@EMNLP (2021)
Bo Zheng, Li Dong, Shaohan Huang, Wenhui Wang, Zewen Chi, Saksham Singhal, Wanxiang Che, Ting Liu, Xia Song, Furu Wei
Consistency Regularization for Cross-Lingual Fine-Tuning. ACL/IJCNLP (1) (2021)
Bo Zheng, Li Dong, Shaohan Huang, Saksham Singhal, Wanxiang Che, Ting Liu, Xia Song, Furu Wei
Allocating Large Vocabulary Capacity for Cross-lingual Language Model Pre-training. CoRR (2021)
Zewen Chi, Li Dong, Furu Wei, Nan Yang, Saksham Singhal, Wenhui Wang, Xia Song, Xian-Ling Mao, Heyan Huang, Ming Zhou
InfoXLM: An Information-Theoretic Framework for Cross-Lingual Language Model Pre-Training. NAACL-HLT (2021)
Damai Dai, Li Dong, Yaru Hao, Zhifang Sui, Furu Wei
Knowledge Neurons in Pretrained Transformers. CoRR (2021)

Li Dong.css-eqlu8i{font-size:small;padding-left:4px;}

Publication Activity (10 Years)

Top Topics

Top Venues

Li Dong