Rongjie Huang

Publication Activity (10 Years)

Years Active: 2017-2024
Publications (10 Years): 82

Top Topics

Diffusion Models

Speech Synthesis

Top Venues

Publications

Dongchao Yang, Songxiang Liu, Rongjie Huang, Chao Weng, Helen Meng
InstructTTS: Modelling Expressive TTS in Discrete Latent Space With Natural Language Style Prompt. IEEE ACM Trans. Audio Speech Lang. Process. 32 (2024)
Xize Cheng, Rongjie Huang, Linjun Li, Zehan Wang, Tao Jin, Aoxiong Yin, Feiyang Chen, Xinyu Duan, Baoxing Huai, Zhou Zhao
TransFace: Unit-Based Audio-Visual Speech Synthesizer for Talking Head Translation. ACL (Findings) (2024)
Zhenhui Ye, Tianyun Zhong, Yi Ren, Jiaqi Yang, Weichuang Li, Jiawei Huang, Ziyue Jiang, Jinzheng He, Rongjie Huang, Jinglin Liu, Chen Zhang, Xiang Yin, Zejun Ma, Zhou Zhao
Real3D-Portrait: One-shot Realistic 3D Talking Portrait Synthesis. ICLR (2024)
Zhiqing Hong, Rongjie Huang, Xize Cheng, Yongqi Wang, Ruiqi Li, Fuming You, Zhou Zhao, Zhimeng Zhang
Text-to-Song: Towards Controllable Music Generation Incorporating Vocals and Accompaniment. CoRR (2024)
Rongjie Huang, Chunlei Zhang, Yongqi Wang, Dongchao Yang, Jinchuan Tian, Zhenhui Ye, Luping Liu, Zehan Wang, Ziyue Jiang, Xuankai Chang, Jiatong Shi, Chao Weng, Zhou Zhao, Dong Yu
Make-A-Voice: Revisiting Voice Large Language Models as Scalable Multilingual and Multitask Learners. ACL (1) (2024)
Huadai Liu, Jialei Wang, Rongjie Huang, Yang Liu, Jiayang Xu, Zhou Zhao
MEDIC: Zero-shot Music Editing with Disentangled Inversion Control. CoRR (2024)
Shengpeng Ji, Jialong Zuo, Minghui Fang, Siqi Zheng, Qian Chen, Wen Wang, Ziyue Jiang, Hai Huang, Xize Cheng, Rongjie Huang, Zhou Zhao
ControlSpeech: Towards Simultaneous Zero-shot Speaker Cloning and Zero-shot Language Style Control With Decoupled Codec. CoRR (2024)
Ruiqi Li, Yu Zhang, Yongqi Wang, Zhiqing Hong, Rongjie Huang, Zhou Zhao
Robust Singing Voice Transcription Serves Synthesis. ACL (1) (2024)
Peng Gao, Le Zhuo, Dongyang Liu, Ruoyi Du, Xu Luo, Longtian Qiu, Yuhang Zhang, Chen Lin, Rongjie Huang, Shijie Geng, Renrui Zhang, Junlin Xi, Wenqi Shao, Zhengkai Jiang, Tianshuo Yang, Weicai Ye, He Tong, Jingwen He, Yu Qiao, Hongsheng Li
Lumina-T2X: Transforming Text into Any Modality, Resolution, and Duration via Flow-based Large Diffusion Transformers. CoRR (2024)
Ruiqi Li, Rongjie Huang, Yongqi Wang, Zhiqing Hong, Zhou Zhao
Self-Supervised Singing Voice Pre-Training towards Speech-to-Singing Conversion. CoRR (2024)
Rongjie Huang, Yusheng Sun, Zhifeng Zhang, Bo Wang, Junxia Ma, Yangyang Chu
Capability Assessment of Cultivating Innovative Talents for Higher Schools Based on Machine Learning. Int. J. Inf. Commun. Technol. Educ. 20 (1) (2024)
Yu Zhang, Rongjie Huang, Ruiqi Li, Jinzheng He, Yan Xia, Feiyang Chen, Xinyu Duan, Baoxing Huai, Zhou Zhao
StyleSinger: Style Transfer for Out-of-Domain Singing Voice Synthesis. AAAI (2024)
Zehan Wang, Ziang Zhang, Hang Zhang, Luping Liu, Rongjie Huang, Xize Cheng, Hengshuang Zhao, Zhou Zhao
OmniBind: Large-scale Omni Multimodal Representation via Binding Spaces. CoRR (2024)
Yongqi Wang, Ruofan Hu, Rongjie Huang, Zhiqing Hong, Ruiqi Li, Wenrui Liu, Fuming You, Tao Jin, Zhou Zhao
Prompt-Singer: Controllable Singing-Voice-Synthesis with Natural Language Prompt. NAACL-HLT (2024)
Ruiqi Li, Rongjie Huang, Yongqi Wang, Zhiqing Hong, Zhou Zhao
Self-Supervised Singing Voice Pre-Training towards Speech-to-Singing Conversion. ACL (Findings) (2024)
Zehan Wang, Ziang Zhang, Xize Cheng, Rongjie Huang, Luping Liu, Zhenhui Ye, Haifeng Huang, Yang Zhao, Tao Jin, Peng Gao, Zhou Zhao
FreeBind: Free Lunch in Unified Multimodal Space via Knowledge Fusion. CoRR (2024)
Ruiqi Li, Zhiqing Hong, Yongqi Wang, Lichao Zhang, Rongjie Huang, Siqi Zheng, Zhou Zhao
Accompanied Singing Voice Synthesis with Fully Text-controlled Melody. CoRR (2024)
Shengpeng Ji, Minghui Fang, Ziyue Jiang, Rongjie Huang, Jialong Zuo, Shulei Wang, Zhou Zhao
Language-Codec: Reducing the Gaps Between Discrete Codec Representation and Speech Language Models. CoRR (2024)
Huadai Liu, Rongjie Huang, Yang Liu, Hengyuan Cao, Jialei Wang, Xize Cheng, Siqi Zheng, Zhou Zhao
AudioLCM: Text-to-Audio Generation with Latent Consistency Models. CoRR (2024)
Ruiqi Li, Yu Zhang, Yongqi Wang, Zhiqing Hong, Rongjie Huang, Zhou Zhao
Robust Singing Voice Transcription Serves Synthesis. CoRR (2024)
Yongqi Wang, Ruofan Hu, Rongjie Huang, Zhiqing Hong, Ruiqi Li, Wenrui Liu, Fuming You, Tao Jin, Zhou Zhao
Prompt-Singer: Controllable Singing-Voice-Synthesis with Natural Language Prompt. CoRR (2024)
Dongchao Yang, Haohan Guo, Yuanyuan Wang, Rongjie Huang, Xiang Li, Xu Tan, Xixin Wu, Helen Meng
UniAudio 1.5: Large Language Model-driven Audio Codec is A Few-shot Audio Task Learner. CoRR (2024)
Huadai Liu, Rongjie Huang, Jinzheng He, Gang Sun, Ran Shen, Xize Cheng, Zhou Zhao
Wav2SQL: Direct Generalizable Speech-To-SQL Parsing. ACL (Findings) (2024)
Zhenhui Ye, Tianyun Zhong, Yi Ren, Jiaqi Yang, Weichuang Li, Jiawei Huang, Ziyue Jiang, Jinzheng He, Rongjie Huang, Jinglin Liu, Chen Zhang, Xiang Yin, Zejun Ma, Zhou Zhao
Real3D-Portrait: One-shot Realistic 3D Talking Portrait Synthesis. CoRR (2024)
Yongqi Wang, Wenxiang Guo, Rongjie Huang, Jiawei Huang, Zehan Wang, Fuming You, Ruiqi Li, Zhou Zhao
Frieren: Efficient Video-to-Audio Generation with Rectified Flow Matching. CoRR (2024)
Zhiqing Hong, Rongjie Huang, Xize Cheng, Yongqi Wang, Ruiqi Li, Fuming You, Zhou Zhao, Zhimeng Zhang
Text-to-Song: Towards Controllable Music Generation Incorporating Vocal and Accompaniment. ACL (1) (2024)
Rongjie Huang, Mingze Li, Dongchao Yang, Jiatong Shi, Xuankai Chang, Zhenhui Ye, Yuning Wu, Zhiqing Hong, Jiawei Huang, Jinglin Liu, Yi Ren, Yuexian Zou, Zhou Zhao, Shinji Watanabe
AudioGPT: Understanding and Generating Speech, Music, Sound, and Talking Head. AAAI (2024)
Le Zhuo, Ruoyi Du, Han Xiao, Yangguang Li, Dongyang Liu, Rongjie Huang, Wenze Liu, Lirui Zhao, Fu-Yun Wang, Zhanyu Ma, Xu Luo, Zehan Wang, Kaipeng Zhang, Xiangyang Zhu, Si Liu, Xiangyu Yue, Dingning Liu, Wanli Ouyang, Ziwei Liu, Yu Qiao, Hongsheng Li, Peng Gao
Lumina-Next: Making Lumina-T2X Stronger and Faster with Next-DiT. CoRR (2024)
Huadai Liu, Rongjie Huang, Xuan Lin, Wenqiang Xu, Maozong Zheng, Hong Chen, Jinzheng He, Zhou Zhao
ViT-TTS: Visual Text-to-Speech with Scalable Diffusion Transformer. EMNLP (2023)
Rongjie Huang, Huadai Liu, Xize Cheng, Yi Ren, Linjun Li, Zhenhui Ye, Jinzheng He, Lichao Zhang, Jinglin Liu, Xiang Yin, Zhou Zhao
AV-TranSpeech: Audio-Visual Robust Speech-to-Speech Translation. ACL (1) (2023)
Xize Cheng, Linjun Li, Tao Jin, Rongjie Huang, Wang Lin, Zehan Wang, Huangdai Liu, Ye Wang, Aoxiong Yin, Zhou Zhao
MixSpeech: Cross-Modality Self-Learning with Audio-Visual Stream Mixup for Visual Speech Translation and Recognition. CoRR (2023)
Rongjie Huang, Jiawei Huang, Dongchao Yang, Yi Ren, Luping Liu, Mingze Li, Zhenhui Ye, Jinglin Liu, Xiang Yin, Zhou Zhao
Make-An-Audio: Text-To-Audio Generation with Prompt-Enhanced Diffusion Models. CoRR (2023)
Haifeng Huang, Zehan Wang, Rongjie Huang, Luping Liu, Xize Cheng, Yang Zhao, Tao Jin, Zhou Zhao
Chat-3D v2: Bridging 3D Scene and Large Language Models with Object Identifiers. CoRR (2023)
Jinzheng He, Jinglin Liu, Zhenhui Ye, Rongjie Huang, Chenye Cui, Huadai Liu, Zhou Zhao
RMSSinger: Realistic-Music-Score based Singing Voice Synthesis. CoRR (2023)
Rongjie Huang, Yi Ren, Ziyue Jiang, Chenye Cui, Jinglin Liu, Zhou Zhao
FastDiff 2: Revisiting and Incorporating GANs and Diffusion Models in High-Fidelity Speech Synthesis. ACL (Findings) (2023)
Zhenhui Ye, Jinzheng He, Ziyue Jiang, Rongjie Huang, Jiawei Huang, Jinglin Liu, Yi Ren, Xiang Yin, Zejun Ma, Zhou Zhao
GeneFace++: Generalized and Stable Real-Time Audio-Driven 3D Talking Face Generation. CoRR (2023)
Jiawei Huang, Yi Ren, Rongjie Huang, Dongchao Yang, Zhenhui Ye, Chen Zhang, Jinglin Liu, Xiang Yin, Zejun Ma, Zhou Zhao
Make-An-Audio 2: Temporal-Enhanced Text-to-Audio Generation. CoRR (2023)
Yongqi Wang, Jionghao Bai, Rongjie Huang, Ruiqi Li, Zhiqing Hong, Zhou Zhao
Speech-to-Speech Translation with Discrete-Unit-Based Style Transfer. CoRR (2023)
Rongjie Huang, Huadai Liu, Xize Cheng, Yi Ren, Linjun Li, Zhenhui Ye, Jinzheng He, Lichao Zhang, Jinglin Liu, Xiang Yin, Zhou Zhao
AV-TranSpeech: Audio-Visual Robust Speech-to-Speech Translation. CoRR (2023)
Chenye Cui, Zhou Zhao, Yi Ren, Jinglin Liu, Rongjie Huang, Feiyang Chen, Zhefeng Wang, Baoxing Huai, Fei Wu
VarietySound: Timbre-Controllable Video to Sound Generation Via Unsupervised Information Disentanglement. ICASSP (2023)
Zhiqing Hong, Chenye Cui, Rongjie Huang, Lichao Zhang, Jinglin Liu, Jinzheng He, Zhou Zhao
UniSinger: Unified End-to-End Singing Voice Synthesis With Cross-Modality Information Matching. ACM Multimedia (2023)
Ruiqi Li, Rongjie Huang, Lichao Zhang, Jinglin Liu, Zhou Zhao
AlignSTS: Speech-to-Singing Conversion via Cross-Modal Alignment. CoRR (2023)
Dongchao Yang, Jinchuan Tian, Xu Tan, Rongjie Huang, Songxiang Liu, Xuankai Chang, Jiatong Shi, Sheng Zhao, Jiang Bian, Xixin Wu, Zhou Zhao, Shinji Watanabe, Helen Meng
UniAudio: An Audio Foundation Model Toward Universal Audio Generation. CoRR (2023)
Huadai Liu, Rongjie Huang, Xuan Lin, Wenqiang Xu, Maozong Zheng, Hong Chen, Jinzheng He, Zhou Zhao
ViT-TTS: Visual Text-to-Speech with Scalable Diffusion Transformer. CoRR (2023)
Rongjie Huang, Mingze Li, Dongchao Yang, Jiatong Shi, Xuankai Chang, Zhenhui Ye, Yuning Wu, Zhiqing Hong, Jiawei Huang, Jinglin Liu, Yi Ren, Zhou Zhao, Shinji Watanabe
AudioGPT: Understanding and Generating Speech, Music, Sound, and Talking Head. CoRR (2023)
Xize Cheng, Tao Jin, Rongjie Huang, Linjun Li, Wang Lin, Zehan Wang, Ye Wang, Huadai Liu, Aoxiong Yin, Zhou Zhao
MixSpeech: Cross-Modality Self-Learning with Audio-Visual Stream Mixup for Visual Speech Translation and Recognition. ICCV (2023)
Huadai Liu, Rongjie Huang, Jinzheng He, Gang Sun, Ran Shen, Xize Cheng, Zhou Zhao
Wav2SQL: Direct Generalizable Speech-To-SQL Parsing. CoRR (2023)
Rongjie Huang, Chunlei Zhang, Yongqi Wang, Dongchao Yang, Luping Liu, Zhenhui Ye, Ziyue Jiang, Chao Weng, Zhou Zhao, Dong Yu
Make-A-Voice: Unified Voice Synthesis With Discrete Representation. CoRR (2023)
Dongchao Yang, Songxiang Liu, Rongjie Huang, Jinchuan Tian, Chao Weng, Yuexian Zou
HiFi-Codec: Group-residual Vector quantization for High Fidelity Audio Codec. CoRR (2023)
Ruiqi Li, Rongjie Huang, Lichao Zhang, Jinglin Liu, Zhou Zhao
AlignSTS: Speech-to-Singing Conversion via Cross-Modal Alignment. ACL (Findings) (2023)
Yu Zhang, Rongjie Huang, Ruiqi Li, Jinzheng He, Yan Xia, Feiyang Chen, Xinyu Duan, Baoxing Huai, Zhou Zhao
StyleSinger: Style Transfer for Out-of-Domain Singing Voice Synthesis. CoRR (2023)
Zhenhui Ye, Rongjie Huang, Yi Ren, Ziyue Jiang, Jinglin Liu, Jinzheng He, Xiang Yin, Zhou Zhao
CLAPSpeech: Learning Prosody from Text Context with Contrastive Language-Audio Pre-Training. ACL (1) (2023)
Ziyue Jiang, Qian Yang, Jialong Zuo, Zhenhui Ye, Rongjie Huang, Yi Ren, Zhou Zhao
FluentSpeech: Stutter-Oriented Automatic Speech Editing with Context-Aware Diffusion Models. CoRR (2023)
Rongjie Huang, Chunlei Zhang, Yi Ren, Zhou Zhao, Dong Yu
Prosody-TTS: Improving Prosody with Masked Autoencoder and Conditional Diffusion Model For Expressive Text-to-Speech. ACL (Findings) (2023)
Jinzheng He, Jinglin Liu, Zhenhui Ye, Rongjie Huang, Chenye Cui, Huadai Liu, Zhou Zhao
RMSSinger: Realistic-Music-Score based Singing Voice Synthesis. ACL (Findings) (2023)
Xize Cheng, Rongjie Huang, Linjun Li, Tao Jin, Zehan Wang, Aoxiong Yin, Minglei Li, Xinyu Duan, Changpeng Yang, Zhou Zhao
TransFace: Unit-Based Audio-Visual Speech Synthesizer for Talking Head Translation. CoRR (2023)
Linjun Li, Tao Jin, Xize Cheng, Ye Wang, Wang Lin, Rongjie Huang, Zhou Zhao
Contrastive Token-Wise Meta-Learning for Unseen Performer Visual Temporal-Aligned Translation. ACL (Findings) (2023)
Zhenhui Ye, Rongjie Huang, Yi Ren, Ziyue Jiang, Jinglin Liu, Jinzheng He, Xiang Yin, Zhou Zhao
CLAPSpeech: Learning Prosody from Text Context with Contrastive Language-Audio Pre-training. CoRR (2023)
Dongchao Yang, Songxiang Liu, Rongjie Huang, Guangzhi Lei, Chao Weng, Helen Meng, Dong Yu
InstructTTS: Modelling Expressive TTS in Discrete Latent Space with Natural Language Style Prompt. CoRR (2023)
Rongjie Huang, Jiawei Huang, Dongchao Yang, Yi Ren, Luping Liu, Mingze Li, Zhenhui Ye, Jinglin Liu, Xiang Yin, Zhou Zhao
Make-An-Audio: Text-To-Audio Generation with Prompt-Enhanced Diffusion Models. ICML (2023)
Ziyue Jiang, Yi Ren, Zhenhui Ye, Jinglin Liu, Chen Zhang, Qian Yang, Shengpeng Ji, Rongjie Huang, Chunfeng Wang, Xiang Yin, Zejun Ma, Zhou Zhao
Mega-TTS: Zero-Shot Text-to-Speech at Scale with Intrinsic Inductive Bias. CoRR (2023)
Ziyue Jiang, Qian Yang, Jialong Zuo, Zhenhui Ye, Rongjie Huang, Yi Ren, Zhou Zhao
FluentSpeech: Stutter-Oriented Automatic Speech Editing with Context-Aware Diffusion Models. ACL (Findings) (2023)
Rongjie Huang, Jinglin Liu, Huadai Liu, Yi Ren, Lichao Zhang, Jinzheng He, Zhou Zhao
TranSpeech: Speech-to-Speech Translation With Bilateral Perturbation. ICLR (2023)
Luping Liu, Zijian Zhang, Yi Ren, Rongjie Huang, Xiang Yin, Zhou Zhao
Detector Guidance for Multi-Object Text-to-Image Generation. CoRR (2023)
Rongjie Huang, Zhou Zhao, Huadai Liu, Jinglin Liu, Chenye Cui, Yi Ren
ProDiff: Progressive Fast Diffusion Model For High-Quality Text-to-Speech. CoRR (2022)
Lichao Zhang, Ruiqi Li, Shoutong Wang, Liqun Deng, Jinglin Liu, Yi Ren, Jinzheng He, Rongjie Huang, Jieming Zhu, Xiao Chen, Zhou Zhao
M4Singer: A Multi-Style, Multi-Singer and Musical Score Provided Mandarin Singing Corpus. NeurIPS (2022)
Rongjie Huang, Chenye Cui, Feiyang Chen, Yi Ren, Jinglin Liu, Zhou Zhao, Baoxing Huai, Zhefeng Wang
SingGAN: Generative Adversarial Network For High-Fidelity Singing Voice Generation. ACM Multimedia (2022)
Rongjie Huang, Zhou Zhao, Huadai Liu, Jinglin Liu, Chenye Cui, Yi Ren
ProDiff: Progressive Fast Diffusion Model for High-Quality Text-to-Speech. ACM Multimedia (2022)
Rongjie Huang, Guizhong Xie, Yudong Zhong, Hongrui Geng, Hao Li, Liangwen Wang
Boundary element analysis of thin structures using a dual transformation method for weakly singular boundary integrals. Comput. Math. Appl. 113 (2022)
Rongjie Huang, Max W. Y. Lam, Jun Wang, Dan Su, Dong Yu, Yi Ren, Zhou Zhao
FastDiff: A Fast Conditional Diffusion Model for High-Quality Speech Synthesis. IJCAI (2022)
Rongjie Huang, Zhou Zhao, Jinglin Liu, Huadai Liu, Yi Ren, Lichao Zhang, Jinzheng He
TranSpeech: Speech-to-Speech Translation With Bilateral Perturbation. CoRR (2022)
Rongjie Huang, Yi Ren, Jinglin Liu, Chenye Cui, Zhou Zhao
GenerSpeech: Towards Style Transfer for Generalizable Out-Of-Domain Text-to-Speech Synthesis. CoRR (2022)
Rongjie Huang, Max W. Y. Lam, Jun Wang, Dan Su, Dong Yu, Yi Ren, Zhou Zhao
FastDiff: A Fast Conditional Diffusion Model for High-Quality Speech Synthesis. CoRR (2022)
Chenye Cui, Yi Ren, Jinglin Liu, Rongjie Huang, Zhou Zhao
VarietySound: Timbre-Controllable Video to Sound Generation via Unsupervised Information Disentanglement. CoRR (2022)
Rongjie Huang, Yi Ren, Jinglin Liu, Chenye Cui, Zhou Zhao
GenerSpeech: Towards Style Transfer for Generalizable Out-Of-Domain Text-to-Speech. NeurIPS (2022)
Rongjie Huang, Feiyang Chen, Yi Ren, Jinglin Liu, Chenye Cui, Zhou Zhao
Multi-Singer: Fast Multi-Singer Singing Voice Vocoder With A Large-Scale Corpus. CoRR (2021)
Chenye Cui, Yi Ren, Jinglin Liu, Feiyang Chen, Rongjie Huang, Ming Lei, Zhou Zhao
EMOVIE: A Mandarin Emotion Speech Dataset with a Simple Emotional Text-to-Speech Model. CoRR (2021)
Feiyang Chen, Rongjie Huang, Chenye Cui, Yi Ren, Jinglin Liu, Zhou Zhao, Nicholas Jing Yuan, Baoxing Huai
SingGAN: Generative Adversarial Network For High-Fidelity Singing Voice Generation. CoRR (2021)
Max W. Y. Lam, Jun Wang, Rongjie Huang, Dan Su, Dong Yu
Bilateral Denoising Diffusion Models. CoRR (2021)
Chenye Cui, Yi Ren, Jinglin Liu, Feiyang Chen, Rongjie Huang, Ming Lei, Zhou Zhao
EMOVIE: A Mandarin Emotion Speech Dataset with a Simple Emotional Text-to-Speech Model. Interspeech (2021)
Rongjie Huang, Feiyang Chen, Yi Ren, Jinglin Liu, Chenye Cui, Zhou Zhao
Multi-Singer: Fast Multi-Singer Singing Voice Vocoder With A Large-Scale Corpus. ACM Multimedia (2021)
Shubin Cai, Rongjie Huang, Ningsheng Yang, Jinwen Jiang, Zhong Ming, Zhengping Liang, Zhiguang Shan
Research on Dynamic Safe Loading Techniques in Android Application Protection System. SmartCom (2017)