Irfan Essa

Publication Activity (10 Years)

Years Active: 2001-2024
Publications (10 Years): 91

Top Topics

Image Generation

Diffusion Models

Human Activity Recognition

Top Venues

Publications

Vincent Cartillier, Neha Jain, Irfan Essa
3D Semantic MapNet: Building Maps for Multi-Object Re-Identification in 3D. CoRR (2024)
Andrew Marmon, Grant Schindler, José Lezama, Dan Kondratyuk, Bryan Seybold, Irfan Essa
CamViG: Camera Aware Image-to-Video Generation with Multimodal Transformers. CoRR (2024)
Harish Haresamudram, Irfan Essa, Thomas Plötz
Towards Learning Discrete Representations via Self-Supervision for Wearables-Based Human Activity Recognition. Sensors 24 (4) (2024)
Apoorva Beedu, Karan Samel, Irfan Essa
On the Efficacy of Text-Based Input Modalities for Action Anticipation. CoRR (2024)
Vincent Cartillier, Grant Schindler, Irfan Essa
SLAIM: Robust Dense Neural SLAM for Online Tracking and Mapping. CoRR (2024)
Lijun Yu, José Lezama, Nitesh Bharadwaj Gundavarapu, Luca Versari, Kihyuk Sohn, David Minnen, Yong Cheng, Agrim Gupta, Xiuye Gu, Alexander G. Hauptmann, Boqing Gong, Ming-Hsuan Yang, Irfan Essa, David A. Ross, Lu Jiang
Language Model Beats Diffusion - Tokenizer is key to visual generation. ICLR (2024)
Seung Hyun Lee, Yinxiao Li, Junjie Ke, Innfarn Yoo, Han Zhang, Jiahui Yu, Qifei Wang, Fei Deng, Glenn Entis, Junfeng He, Gang Li, Sangpil Kim, Irfan Essa, Feng Yang
Parrot: Pareto-optimal Multi-Reward Reinforcement Learning Framework for Text-to-Image Generation. CoRR (2024)
K. Niranjan Kumar, Irfan Essa, Sehoon Ha
Words into Action: Learning Diverse Humanoid Robot Behaviors using Language Guided Iterative Motion Refinement. CoRR (2023)
Harish Haresamudram, Irfan Essa, Thomas Plötz
Investigating Enhancements to Contrastive Predictive Coding for Human Activity Recognition. PERCOM (2023)
Dina Bashkirova, José Lezama, Kihyuk Sohn, Kate Saenko, Irfan Essa
MaskSketch: Unpaired Structure-guided Masked Image Generation. CVPR (2023)
Nikolai Warner, Meera Hahn, Jonathan Huang, Irfan Essa, Vighnesh Birodkar
Text and Click inputs for unambiguous open vocabulary instance segmentation. CoRR (2023)
Tianle Huang, Nitish Sontakke, K. Niranjan Kumar, Irfan Essa, Stefanos Nikolaidis, Dennis W. Hong, Sehoon Ha
BayRnTune: Adaptive Bayesian Domain Randomization via Strategic Fine-tuning. CoRR (2023)
Lijun Yu, José Lezama, Nitesh Bharadwaj Gundavarapu, Luca Versari, Kihyuk Sohn, David Minnen, Yong Cheng, Agrim Gupta, Xiuye Gu, Alexander G. Hauptmann, Boqing Gong, Ming-Hsuan Yang, Irfan Essa, David A. Ross, Lu Jiang
Language Model Beats Diffusion - Tokenizer is Key to Visual Generation. CoRR (2023)
Harish Haresamudram, Irfan Essa, Thomas Ploetz
Towards Learning Discrete Representations via Self-Supervision for Wearables-Based Human Activity Recognition. CoRR (2023)
Agrim Gupta, Lijun Yu, Kihyuk Sohn, Xiuye Gu, Meera Hahn, Li Fei-Fei, Irfan Essa, Lu Jiang, José Lezama
Photorealistic Video Generation with Diffusion Models. CoRR (2023)
Dan Kondratyuk, Lijun Yu, Xiuye Gu, José Lezama, Jonathan Huang, Rachel Hornung, Hartwig Adam, Hassan Akbari, Yair Alon, Vighnesh Birodkar, Yong Cheng, Ming-Chang Chiu, Josh Dillon, Irfan Essa, Agrim Gupta, Meera Hahn, Anja Hauth, David Hendon, Alonso Martinez, David Minnen, David A. Ross, Grant Schindler, Mikhail Sirotenko, Kihyuk Sohn, Krishna Somandepalli, Huisheng Wang, Jimmy Yan, Ming-Hsuan Yang, Xuan Yang, Bryan Seybold, Lu Jiang
VideoPoet: A Large Language Model for Zero-Shot Video Generation. CoRR (2023)
Vighnesh Birodkar, Jonathan Huang, Meera Hahn, Irfan Essa, Nikolai Warner
Text and Click inputs for unambiguous open vocabulary instance segmentation. BMVC (2023)
K. Niranjan Kumar, Irfan Essa, Sehoon Ha
Cascaded Compositional Residual Learning for Complex Interactive Behaviors. IEEE Robotics Autom. Lett. 8 (8) (2023)
Lijun Yu, Yong Cheng, Zhiruo Wang, Vivek Kumar, Wolfgang Macherey, Yanping Huang, David A. Ross, Irfan Essa, Yonatan Bisk, Ming-Hsuan Yang, Kevin Murphy, Alexander G. Hauptmann, Lu Jiang
SPAE: Semantic Pyramid AutoEncoder for Multimodal Generation with Frozen LLMs. CoRR (2023)
Lijun Yu, Yong Cheng, Kihyuk Sohn, José Lezama, Han Zhang, Huiwen Chang, Alexander G. Hauptmann, Ming-Hsuan Yang, Yuan Hao, Irfan Essa, Lu Jiang
MAGVIT: Masked Generative Video Transformer. CVPR (2023)
José Lezama, Tim Salimans, Lu Jiang, Huiwen Chang, Jonathan Ho, Irfan Essa
Discrete Predictor-Corrector Diffusion Models for Image Synthesis. ICLR (2023)
Daniel Nkemelu, Peggy Chi, Daniel Castro Chin, Krishna Srinivasan, Irfan Essa
Automatic Multi-Path Web Story Creation from a Structural Article. CoRR (2023)
Kihyuk Sohn, Albert Shaw, Yuan Hao, Han Zhang, Luisa Polania, Huiwen Chang, Lu Jiang, Irfan Essa
Learning Disentangled Prompts for Compositional Image Synthesis. CoRR (2023)
Dina Bashkirova, José Lezama, Kihyuk Sohn, Kate Saenko, Irfan Essa
MaskSketch: Unpaired Structure-guided Masked Image Generation. CoRR (2023)
Kihyuk Sohn, Lu Jiang, Jarred Barber, Kimin Lee, Nataniel Ruiz, Dilip Krishnan, Huiwen Chang, Yuanzhen Li, Irfan Essa, Michael Rubinstein, Yuan Hao, Glenn Entis, Irina Blok, Daniel Castro Chin
StyleDrop: Text-to-Image Synthesis of Any Style. NeurIPS (2023)
Erik Wijmans, Manolis Savva, Irfan Essa, Stefan Lee, Ari S. Morcos, Dhruv Batra
Emergence of Maps in the Memories of Blind Navigation Agents. CoRR (2023)
Karan Samel, Jun Ma, Zhengyang Wang, Tong Zhao, Irfan Essa
Integrating Noisy Knowledge into Language Representations for E-Commerce Applications. BigData (2023)
Daniel Nkemelu, Harshil Shah, Irfan Essa, Michael L. Best
Tackling Hate Speech in Low-resource Languages with Context Experts. CoRR (2023)
Hyeongju Choi, Apoorva Beedu, Irfan Essa
Multimodal Contrastive Learning with Hard Negative Sampling for Human Activity Recognition. CoRR (2023)
Erik Wijmans, Manolis Savva, Irfan Essa, Stefan Lee, Ari S. Morcos, Dhruv Batra
Emergence of Maps in the Memories of Blind Navigation Agents. ICLR (2023)
Lijun Yu, Yong Cheng, Zhiruo Wang, Vivek Kumar, Wolfgang Macherey, Yanping Huang, David A. Ross, Irfan Essa, Yonatan Bisk, Ming-Hsuan Yang, Kevin P. Murphy, Alexander G. Hauptmann, Lu Jiang
SPAE: Semantic Pyramid AutoEncoder for Multimodal Generation with Frozen LLMs. NeurIPS (2023)
Xingqian Xu, Jiayi Guo, Zhangyang Wang, Gao Huang, Irfan Essa, Humphrey Shi
Prompt-Free Diffusion: Taking "Text" out of Text-to-Image Diffusion Models. CoRR (2023)
Kihyuk Sohn, Nataniel Ruiz, Kimin Lee, Daniel Castro Chin, Irina Blok, Huiwen Chang, Jarred Barber, Lu Jiang, Glenn Entis, Yuanzhen Li, Yuan Hao, Irfan Essa, Michael Rubinstein, Dilip Krishnan
StyleDrop: Text-to-Image Generation in Any Style. CoRR (2023)
Erik Wijmans, Manolis Savva, Irfan Essa, Stefan Lee, Ari S. Morcos, Dhruv Batra
Emergence of Maps in the Memories of Blind Navigation Agents. AI Matters 9 (2) (2023)
Yi-Hao Peng, Peggy Chi, Anjuli Kannan, Meredith Ringel Morris, Irfan Essa
Slide Gestalt: Automatic Structure Extraction in Slide Decks for Non-Visual Access. CHI (2023)
Kihyuk Sohn, Huiwen Chang, José Lezama, Luisa Polania, Han Zhang, Yuan Hao, Irfan Essa, Lu Jiang
Visual Prompt Tuning for Generative Transfer Learning. CVPR (2023)
Harish Haresamudram, Irfan Essa, Thomas Plötz
Assessing the State of Self-Supervised Human Activity Recognition Using Wearables. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 6 (3) (2022)
K. Niranjan Kumar, Irfan Essa, Sehoon Ha
Graph-based Cluttered Scene Generation and Interactive Exploration using Deep Reinforcement Learning. ICRA (2022)
Erik Wijmans, Irfan Essa, Dhruv Batra
VER: Scaling On-Policy RL Leads to the Emergence of Navigation in Embodied Rearrangement. NeurIPS (2022)
Peggy Chi, Tao Dong, Christian Früh, Brian Colonna, Vivek Kwatra, Irfan Essa
Synthesis-Assisted Video Prototyping From a Document. UIST (2022)
José Lezama, Huiwen Chang, Lu Jiang, Irfan Essa
Improved Masked Image Generation with Token-Critic. ECCV (23) (2022)
Erik Wijmans, Irfan Essa, Dhruv Batra
How to Train PointGoal Navigation Agents on a (Sample and Compute) Budget. AAMAS (2022)
Xiang Kong, Lu Jiang, Huiwen Chang, Han Zhang, Yuan Hao, Haifeng Gong, Irfan Essa
BLT: Bidirectional Layout Transformer for Controllable Layout Generation. ECCV (17) (2022)
Karan Samel, Zelin Zhao, Binghong Chen, Shuang Li, Dharmashankar Subramanian, Irfan Essa, Le Song
Learning Temporal Rules from Noisy Timeseries Data. CoRR (2022)
Harish Haresamudram, Irfan Essa, Thomas Plötz
Assessing the State of Self-Supervised Human Activity Recognition using Wearables. CoRR (2022)
Apoorva Beedu, Huda AlAmri, Irfan Essa
Video based Object 6D Pose Estimation using Transformers. CoRR (2022)
Chengzhi Mao, Lu Jiang, Mostafa Dehghani, Carl Vondrick, Rahul Sukthankar, Irfan Essa
Discrete Representations Strengthen Vision Transformer Robustness. ICLR (2022)
Lijun Yu, Yong Cheng, Kihyuk Sohn, José Lezama, Han Zhang, Huiwen Chang, Alexander G. Hauptmann, Ming-Hsuan Yang, Yuan Hao, Irfan Essa, Lu Jiang
MAGVIT: Masked Generative Video Transformer. CoRR (2022)
Kihyuk Sohn, Yuan Hao, José Lezama, Luisa Polania, Huiwen Chang, Han Zhang, Irfan Essa, Lu Jiang
Visual Prompt Tuning for Generative Transfer Learning. CoRR (2022)
José Lezama, Huiwen Chang, Lu Jiang, Irfan Essa
Improved Masked Image Generation with Token-Critic. CoRR (2022)
Harish Haresamudram, Irfan Essa, Thomas Ploetz
Investigating Enhancements to Contrastive Predictive Coding for Human Activity Recognition. CoRR (2022)
Huda AlAmri, Anthony Bilic, Michael Hu, Apoorva Beedu, Irfan Essa
End-to-End Multimodal Representation Learning for Video Dialog. CoRR (2022)
Hyeongju Choi, Apoorva Beedu, Harish Haresamudram, Irfan Essa
Multi-Stage Based Feature Fusion of Multi-Modal Data for Human Activity Recognition. CoRR (2022)
Erik Wijmans, Irfan Essa, Dhruv Batra
VER: Scaling On-Policy RL Leads to the Emergence of Navigation in Embodied Rearrangement. CoRR (2022)
Daniel Scarafoni, Irfan Essa, Thomas Ploetz
Finding Islands of Predictability in Action Forecasting. CoRR (2022)
K. Niranjan Kumar, Irfan Essa, Sehoon Ha
Cascaded Compositional Residual Learning for Complex Interactive Behaviors. CoRR (2022)
Peggy Chi, Nathan Frey, Katrina Panovich, Irfan Essa
Automatic Instructional Video Creation from a Markdown-Formatted Tutorial. UIST (2021)
Nathan Frey, Peggy Chi, Weilong Yang, Irfan Essa
Automatic Non-Linear Video Editing Transfer. CoRR (2021)
Chengzhi Mao, Lu Jiang, Mostafa Dehghani, Carl Vondrick, Rahul Sukthankar, Irfan Essa
Discrete Representations Strengthen Vision Transformer Robustness. CoRR (2021)
A. J. Piergiovanni, Anelia Angelova, Michael S. Ryoo, Irfan Essa
Unsupervised Discovery of Actions in Instructional Videos. BMVC (2021)
Dan Scarafoni, Irfan Essa, Thomas Ploetz
PLAN-B: Predicting Likely Alternative Next Best Sequences for Action Prediction. CoRR (2021)
Xiang Kong, Lu Jiang, Huiwen Chang, Han Zhang, Yuan Hao, Haifeng Gong, Irfan Essa
BLT: Bidirectional Layout Transformer for Controllable Layout Generation. CoRR (2021)
Tianhao Zhang, Hung-Yu Tseng, Lu Jiang, Weilong Yang, Honglak Lee, Irfan Essa
Text as Neural Operator: Image Manipulation by Text Instruction. ACM Multimedia (2021)
K. Niranjan Kumar, Irfan Essa, Sehoon Ha
Graph-based Cluttered Scene Generation and Interactive Exploration using Deep Reinforcement Learning. CoRR (2021)
Vincent Cartillier, Zhile Ren, Neha Jain, Stefan Lee, Irfan Essa, Dhruv Batra
Semantic MapNet: Building Allocentric Semantic Maps and Representations from Egocentric Views. AAAI (2021)
Apoorva Beedu, Zhile Ren, Varun Agrawal, Irfan Essa
VideoPose: Estimating 6D object pose from videos. CoRR (2021)
Anh Truong, Peggy Chi, David Salesin, Irfan Essa, Maneesh Agrawala
Automatic Generation of Two-Level Hierarchical Tutorials from Instructional Makeup Videos. CHI (2021)
Vincent Cartillier, Zhile Ren, Neha Jain, Stefan Lee, Irfan Essa, Dhruv Batra
Semantic MapNet: Building Allocentric SemanticMaps and Representations from Egocentric Views. CoRR (2020)
Erik Wijmans, Abhishek Kadian, Ari Morcos, Stefan Lee, Irfan Essa, Devi Parikh, Manolis Savva, Dhruv Batra
DD-PPO: Learning Near-Perfect PointGoal Navigators from 2.5 Billion Frames. ICLR (2020)
Tianhao Zhang, Hung-Yu Tseng, Lu Jiang, Honglak Lee, Irfan Essa, Weilong Yang
Text as Neural Operator: Image Manipulation by Text Instruction. CoRR (2020)
Hsin-Ying Lee, Lu Jiang, Irfan Essa, Phuong B. Le, Haifeng Gong, Ming-Hsuan Yang, Weilong Yang
Neural Design Network: Graphic Layout Generation with Constraints. ECCV (3) (2020)
Erik Wijmans, Irfan Essa, Dhruv Batra
How to Train PointGoal Navigation Agents on a (Sample and Compute) Budget. CoRR (2020)
Erik Wijmans, Julian Straub, Dhruv Batra, Irfan Essa, Judy Hoffman, Ari Morcos
Analyzing Visual Representations in Embodied Navigation Tasks. CoRR (2020)
Peggy Chi, Zheng Sun, Katrina Panovich, Irfan Essa
Automatic Video Creation From a Web Page. UIST (2020)
Huda AlAmri, Vincent Cartillier, Abhishek Das, Jue Wang, Stefan Lee, Peter Anderson, Irfan Essa, Devi Parikh, Dhruv Batra, Anoop Cherian, Tim K. Marks, Chiori Hori
Audio-Visual Scene-Aware Dialog. CoRR (2019)
Chiori Hori, Huda AlAmri, Jue Wang, Gordon Wichern, Takaaki Hori, Anoop Cherian, Tim K. Marks, Vincent Cartillier, Raphael Gontijo Lopes, Abhishek Das, Irfan Essa, Dhruv Batra, Devi Parikh
End-to-end Audio Visual Scene-aware Dialog Using Multimodal Attention-based Video Features. ICASSP (2019)
Hsin-Ying Lee, Weilong Yang, Lu Jiang, Madison Le, Irfan Essa, Haifeng Gong, Ming-Hsuan Yang
Neural Design Network: Graphic Layout Generation with Constraints. CoRR (2019)
Niranjan Kumar Kannabiran, Irfan Essa, C. Karen Liu
Estimating Mass Distribution of Articulated Objects through Physical Interaction. CoRR (2019)
Huda AlAmri, Vincent Cartillier, Abhishek Das, Jue Wang, Anoop Cherian, Irfan Essa, Dhruv Batra, Tim K. Marks, Chiori Hori, Peter Anderson, Stefan Lee, Devi Parikh
Audio Visual Scene-Aware Dialog. CVPR (2019)
Erik Wijmans, Samyak Datta, Oleksandr Maksymets, Abhishek Das, Georgia Gkioxari, Stefan Lee, Irfan Essa, Devi Parikh, Dhruv Batra
Embodied Question Answering in Photorealistic Environments with Point Cloud Perception. CoRR (2019)
Erik Wijmans, Abhishek Kadian, Ari Morcos, Stefan Lee, Irfan Essa, Devi Parikh, Manolis Savva, Dhruv Batra
Decentralized Distributed PPO: Solving PointGoal Navigation. CoRR (2019)
Erik Wijmans, Samyak Datta, Oleksandr Maksymets, Abhishek Das, Georgia Gkioxari, Stefan Lee, Irfan Essa, Devi Parikh, Dhruv Batra
Embodied Question Answering in Photorealistic Environments With Point Cloud Perception. CVPR (2019)
Luke Drnach, Jessica L. Allen, Irfan Essa, Lena H. Ting
A Data-Driven Predictive Model of Individual-Specific Effects of FES on Human Gait Dynamics. ICRA (2019)
Jonathan C. Balloch, Varun Agrawal, Irfan Essa, Sonia Chernova
Unbiasing Semantic Segmentation For Robot Perception using Synthetic Data Feature Transfer. CoRR (2018)
Luke Drnach, Irfan Essa, Lena H. Ting
Identifying Gait Phases from Joint Kinematics during Walking with Switched Linear Dynamical Systems. BioRob (2018)
Erkam Uzun, Simon Pak Ho Chung, Irfan Essa, Wenke Lee
rtCaptcha: A Real-Time CAPTCHA Based Liveness Detection System. NDSS (2018)
Chiori Hori, Huda AlAmri, Jue Wang, Gordon Wichern, Takaaki Hori, Anoop Cherian, Tim K. Marks, Vincent Cartillier, Raphael Gontijo Lopes, Abhishek Das, Irfan Essa, Dhruv Batra, Devi Parikh
End-to-End Audio Visual Scene-Aware Dialog using Multimodal Attention-Based Video Features. CoRR (2018)
Huda AlAmri, Vincent Cartillier, Raphael Gontijo Lopes, Abhishek Das, Jue Wang, Irfan Essa, Dhruv Batra, Devi Parikh, Anoop Cherian, Tim K. Marks, Chiori Hori
Audio Visual Scene-Aware Dialog (AVSD) Challenge at DSTC7. CoRR (2018)
Amirreza Shaban, Shray Bansal, Zhen Liu, Irfan Essa, Byron Boots
One-Shot Learning for Semantic Segmentation. BMVC (2017)
Thomas B. Moeslund, Graham A. Thomas, Adrian Hilton, Peter Carr, Irfan Essa
Computer Vision in Sports. Comput. Vis. Image Underst. 159 (2017)
Amirreza Shaban, Shray Bansal, Zhen Liu, Irfan Essa, Byron Boots
One-Shot Learning for Semantic Segmentation. CoRR (2017)
Scott Stillman, Irfan Essa
Towards reliable multimodal sensing in aware environments. PUI (2001)