Puyuan Peng

Publication Activity (10 Years)

Years Active: 2020-2024
Publications (10 Years): 26

Top Topics

Pointwise Mutual Information

Visual Representations

Top Venues

Publications

Yuan Tseng, Layne Berry, Yiting Chen, I-Hsiang Chiu, Hsuan-Hao Lin, Max Liu, Puyuan Peng, Yi-Jen Shih, Hung-Yu Wang, Haibin Wu, Poyao Huang, Chun-Mao Lai, Shang-Wen Li, David Harwath, Yu Tsao, Abdelrahman Mohamed, Chi-Luen Feng, Hung-Yi Lee
AV-SUPERB: A Multi-Task Evaluation Benchmark for Audio-Visual Representation Models. ICASSP (2024)
Changan Chen, Puyuan Peng, Ami Baid, Zihui Xue, Wei-Ning Hsu, David Harwath, Kristen Grauman
Action2Sound: Ambient-Aware Generation of Action Sounds from Egocentric Videos. CoRR (2024)
Hsuan-Fu Wang, Yi-Jen Shih, Heng-Jui Chang, Layne Berry, Puyuan Peng, Hung-yi Lee, Hsin-Min Wang, David Harwath
SpeechCLIP+: Self-supervised multi-task representation learning for speech via CLIP and speech-image data. CoRR (2024)
Zhisheng Zheng, Puyuan Peng, Ziyang Ma, Xie Chen, Eunsol Choi, David Harwath
BAT: Learning to Reason about Spatial Sounds with Large Language Models. CoRR (2024)
Puyuan Peng, Po-Yao Huang, Shang-Wen Li, Abdelrahman Mohamed, David Harwath
VoiceCraft: Zero-Shot Speech Editing and Text-to-Speech in the Wild. ACL (1) (2024)
Puyuan Peng, Po-Yao Huang, Daniel Li, Abdelrahman Mohamed, David Harwath
VoiceCraft: Zero-Shot Speech Editing and Text-to-Speech in the Wild. CoRR (2024)
Hung-Chieh Fang, Nai-Xuan Ye, Yi-Jen Shih, Puyuan Peng, Hsuan-Fu Wang, Layne Berry, Hung-yi Lee, David Harwath
Integrating Self-supervised Speech Model with Pseudo Word-level Targets from Visually-grounded Speech Model. CoRR (2024)
Puyuan Peng, Brian Yan, Shinji Watanabe, David Harwath
Prompting the Hidden Talent of Web-Scale Speech Models for Zero-Shot Task Generalization. INTERSPEECH (2023)
Cheng-I Jeff Lai, Freda Shi, Puyuan Peng, Yoon Kim, Kevin Gimpel, Shiyu Chang, Yung-Sung Chuang, Saurabhchand Bhati, David D. Cox, David Harwath, Yang Zhang, Karen Livescu, James R. Glass
Audio-Visual Neural Syntax Acquisition. CoRR (2023)
Chiori Hori, Puyuan Peng, David Harwath, Xinyu Liu, Kei Ota, Siddarth Jain, Radu Corcodel, Devesh K. Jha, Diego Romeres, Jonathan Le Roux
Style-transfer based Speech and Audio-visual Scene Understanding for Robot Action Sequence Acquisition from Videos. CoRR (2023)
Puyuan Peng, Brian Yan, Shinji Watanabe, David Harwath
Prompting the Hidden Talent of Web-Scale Speech Models for Zero-Shot Task Generalization. CoRR (2023)
Yuan Tseng, Layne Berry, Yi-Ting Chen, I-Hsiang Chiu, Hsuan-Hao Lin, Max Liu, Puyuan Peng, Yi-Jen Shih, Hung-Yu Wang, Haibin Wu, Po-Yao Huang, Chun-Mao Lai, Shang-Wen Li, David Harwath, Yu Tsao, Shinji Watanabe, Abdelrahman Mohamed, Chi-Luen Feng, Hung-yi Lee
AV-SUPERB: A Multi-Task Evaluation Benchmark for Audio-Visual Representation Models. CoRR (2023)
Puyuan Peng, Shang-Wen Li, Okko Räsänen, Abdelrahman Mohamed, David Harwath
Syllable Discovery and Cross-Lingual Generalization in a Visually Grounded, Self-Supervised Speech Model. INTERSPEECH (2023)
Puyuan Peng, Shang-Wen Li, Okko Räsänen, Abdelrahman Mohamed, David Harwath
Syllable Discovery and Cross-Lingual Generalization in a Visually Grounded, Self-Supervised Speech Mode. CoRR (2023)
Cheng-I Jeff Lai, Freda Shi, Puyuan Peng, Yoon Kim, Kevin Gimpel, Shiyu Chang, Yung-Sung Chuang, Saurabhchand Bhati, David D. Cox, David Harwath, Yang Zhang, Karen Livescu, James R. Glass
Audio-Visual Neural Syntax Acquisition. ASRU (2023)
Chiori Hori, Puyuan Peng, David Harwath, Xinyu Liu, Kei Ota, Siddarth Jain, Radu Corcodel, Devesh K. Jha, Diego Romeres, Jonathan Le Roux
Style-transfer based Speech and Audio-visual Scene understanding for Robot Action Sequence Acquisition from Videos. INTERSPEECH (2023)
Anuj Diwan, Puyuan Peng, Raymond J. Mooney
Zero-shot Video Moment Retrieval With Off-the-Shelf Models. TL4NLP (2022)
Alan Baade, Puyuan Peng, David Harwath
MAE-AST: Masked Autoencoding Audio Spectrogram Transformer. INTERSPEECH (2022)
Alan Baade, Puyuan Peng, David Harwath
MAE-AST: Masked Autoencoding Audio Spectrogram Transformer. CoRR (2022)
Puyuan Peng, David Harwath
Word Discovery in Visually Grounded, Self-Supervised Speech Models. CoRR (2022)
Anuj Diwan, Puyuan Peng, Raymond J. Mooney
Zero-shot Video Moment Retrieval With Off-the-Shelf Models. CoRR (2022)
Puyuan Peng, David Harwath
Fast-Slow Transformer for Visually Grounding Speech. ICASSP (2022)
Puyuan Peng, David Harwath
Word Discovery in Visually Grounded, Self-Supervised Speech Models. INTERSPEECH (2022)
Puyuan Peng, David Harwath
Self-Supervised Representation Learning for Speech Using Visual Grounding and Masked Language Modeling. CoRR (2022)
Puyuan Peng, David Harwath
Fast-Slow Transformer for Visually Grounding Speech. CoRR (2021)
Puyuan Peng, Herman Kamper, Karen Livescu
A Correspondence Variational Autoencoder for Unsupervised Acoustic Word Embeddings. CoRR (2020)