Masatoshi Uehara

Publication Activity (10 Years)

Years Active: 2018-2024
Publications (10 Years): 72

Top Topics

Markov Decision Process

Reinforcement Learning

Diffusion Models

Partially Observable Markov Decision Process

Top Venues

Publications

Jakub Grudzien Kuba, Masatoshi Uehara, Pieter Abbeel, Sergey Levine
Functional Graphical Models: Structure Enables Offline Data-Driven Optimization. CoRR (2024)
Yulai Zhao, Masatoshi Uehara, Gabriele Scalia, Tommaso Biancalani, Sergey Levine, Ehsan Hajiramezanali
Adding Conditional Control to Diffusion Models with Reinforcement Learning. CoRR (2024)
Wenhao Zhan, Masatoshi Uehara, Wen Sun, Jason D. Lee
Provable Reward-Agnostic Preference-Based Reinforcement Learning. ICLR (2024)
Masatoshi Uehara, Yulai Zhao, Tommaso Biancalani, Sergey Levine
Understanding Reinforcement Learning-Based Fine-Tuning of Diffusion Models: A Tutorial and Review. CoRR (2024)
Kuba Grudzien Kuba, Masatoshi Uehara, Sergey Levine, Pieter Abbeel
Functional Graphical Models: Structure Enables Offline Data-Driven Optimization. AISTATS (2024)
Masatoshi Uehara, Yulai Zhao, Kevin Black, Ehsan Hajiramezanali, Gabriele Scalia, Nathaniel Lee Diamant, Alex M. Tseng, Tommaso Biancalani, Sergey Levine
Fine-Tuning of Continuous-Time Diffusion Models as Entropy-Regularized Control. CoRR (2024)
Masatoshi Uehara, Yulai Zhao, Kevin Black, Ehsan Hajiramezanali, Gabriele Scalia, Nathaniel Lee Diamant, Alex M. Tseng, Sergey Levine, Tommaso Biancalani
Feedback Efficient Online Fine-Tuning of Diffusion Models. CoRR (2024)
Wenhao Zhan, Masatoshi Uehara, Nathan Kallus, Jason D. Lee, Wen Sun
Provable Offline Preference-Based Reinforcement Learning. ICLR (2024)
Zihao Li, Hui Lan, Vasilis Syrgkanis, Mengdi Wang, Masatoshi Uehara
Regularized DeepIV with Model Selection. CoRR (2024)
Masatoshi Uehara, Yulai Zhao, Ehsan Hajiramezanali, Gabriele Scalia, Gökcen Eraslan, Avantika Lal, Sergey Levine, Tommaso Biancalani
Bridging Model-Based Optimization and Generative Modeling via Conservative Fine-Tuning of Diffusion Models. CoRR (2024)
Masatoshi Uehara, Nathan Kallus, Jason D. Lee, Wen Sun
Refined Value-Based Offline RL under Realizability and Partial Coverage. CoRR (2023)
Haruka Kiyohara, Masatoshi Uehara, Yusuke Narita, Nobuyuki Shimizu, Yasuo Yamamoto, Yuta Saito
Off-Policy Evaluation of Ranking Policies under Diverse User Behavior. CoRR (2023)
Masatoshi Uehara, Nathan Kallus, Jason D. Lee, Wen Sun
Offline Minimax Soft-Q-learning Under Realizability and Partial Coverage. NeurIPS (2023)
Wenhao Zhan, Masatoshi Uehara, Nathan Kallus, Jason D. Lee, Wen Sun
Provable Offline Reinforcement Learning with Human Feedback. CoRR (2023)
Runzhe Wu, Masatoshi Uehara, Wen Sun
Distributional Offline Policy Evaluation with Predictive Error Guarantees. CoRR (2023)
Andrew Bennett, Nathan Kallus, Xiaojie Mao, Whitney Newey, Vasilis Syrgkanis, Masatoshi Uehara
Convergence Guarantees without Identification or Closedness. COLT (2023)
Andrew Bennett, Nathan Kallus, Xiaojie Mao, Whitney Newey, Vasilis Syrgkanis, Masatoshi Uehara
Inference on Strongly Identified Functionals of Weakly Identified Functions. COLT (2023)
Runzhe Wu, Masatoshi Uehara, Wen Sun
Distributional Offline Policy Evaluation with Predictive Error Guarantees. ICML (2023)
Andrew Bennett, Nathan Kallus, Xiaojie Mao, Whitney Newey, Vasilis Syrgkanis, Masatoshi Uehara
Source Condition Double Robust Inference on Functionals of Inverse Problems. CoRR (2023)
Andrew Bennett, Nathan Kallus, Xiaojie Mao, Whitney Newey, Vasilis Syrgkanis, Masatoshi Uehara
Minimax Instrumental Variable Regression and $L_2$ Convergence Guarantees without Identification or Closedness. CoRR (2023)
Haruka Kiyohara, Masatoshi Uehara, Yusuke Narita, Nobuyuki Shimizu, Yasuo Yamamoto, Yuta Saito
Off-Policy Evaluation of Ranking Policies under Diverse User Behavior. KDD (2023)
Wenhao Zhan, Masatoshi Uehara, Wen Sun, Jason D. Lee
PAC Reinforcement Learning for Predictive State Representations. ICLR (2023)
Masatoshi Uehara, Haruka Kiyohara, Andrew Bennett, Victor Chernozhukov, Nan Jiang, Nathan Kallus, Chengchun Shi, Wen Sun
Future-Dependent Value-Based Off-Policy Evaluation in POMDPs. NeurIPS (2023)
Wenhao Zhan, Masatoshi Uehara, Wen Sun, Jason D. Lee
How to Query Human Feedback Efficiently in RL? CoRR (2023)
Masatoshi Uehara, Ayush Sekhari, Jason D. Lee, Nathan Kallus, Wen Sun
Computationally Efficient PAC RL in POMDPs with Latent Determinism and Conditional Embeddings. ICML (2023)
Wenhao Zhan, Masatoshi Uehara, Wen Sun, Jason D. Lee
PAC Reinforcement Learning for Predictive State Representations. CoRR (2022)
Nathan Kallus, Masatoshi Uehara
Efficiently Breaking the Curse of Horizon in Off-Policy Evaluation with Double Reinforcement Learning. Oper. Res. 70 (6) (2022)
Masatoshi Uehara, Xuezhou Zhang, Wen Sun
Representation Learning for Online and Offline RL in Low-rank MDPs. ICLR (2022)
Masatoshi Uehara, Ayush Sekhari, Jason D. Lee, Nathan Kallus, Wen Sun
Computationally Efficient PAC RL in POMDPs with Latent Determinism and Conditional Embeddings. CoRR (2022)
Xuezhou Zhang, Yuda Song, Masatoshi Uehara, Mengdi Wang, Alekh Agarwal, Wen Sun
Efficient Reinforcement Learning in Block MDPs: A Model-free Representation Learning Approach. CoRR (2022)
Xuezhou Zhang, Yuda Song, Masatoshi Uehara, Mengdi Wang, Alekh Agarwal, Wen Sun
Efficient Reinforcement Learning in Block MDPs: A Model-free Representation Learning approach. ICML (2022)
Masatoshi Uehara, Haruka Kiyohara, Andrew Bennett, Victor Chernozhukov, Nan Jiang, Nathan Kallus, Chengchun Shi, Wen Sun
Future-Dependent Value-Based Off-Policy Evaluation in POMDPs. CoRR (2022)
Masatoshi Uehara, Ayush Sekhari, Jason D. Lee, Nathan Kallus, Wen Sun
Provably Efficient Reinforcement Learning in Partially Observable Dynamical Systems. NeurIPS (2022)
Masahiro Kato, Kaito Ariu, Masaaki Imaizumi, Masatoshi Uehara, Masahiro Nomura, Chao Qin
Optimal Fixed-Budget Best Arm Identification using the Augmented Inverse Probability Weighting Estimator in Two-Armed Gaussian Bandits with Unknown Variances. CoRR (2022)
Masatoshi Uehara, Ayush Sekhari, Jason D. Lee, Nathan Kallus, Wen Sun
Provably Efficient Reinforcement Learning in Partially Observable Dynamical Systems. CoRR (2022)
Masatoshi Uehara, Wen Sun
Pessimistic Model-based Offline Reinforcement Learning under Partial Coverage. ICLR (2022)
Masatoshi Uehara, Chengchun Shi, Nathan Kallus
A Review of Off-Policy Evaluation in Reinforcement Learning. CoRR (2022)
Chengchun Shi, Masatoshi Uehara, Jiawei Huang, Nan Jiang
A Minimax Learning Approach to Off-Policy Evaluation in Confounded Partially Observable Markov Decision Processes. ICML (2022)
Nathan Kallus, Xiaojie Mao, Masatoshi Uehara
Causal Inference Under Unmeasured Confounding With Negative Controls: A Minimax Learning Approach. CoRR (2021)
Takeru Matsuda, Masatoshi Uehara, Aapo Hyvärinen
Information criteria for non-normalized models. J. Mach. Learn. Res. 22 (2021)
Yichun Hu, Nathan Kallus, Masatoshi Uehara
Fast Rates for the Regret of Offline Reinforcement Learning. CoRR (2021)
Masatoshi Uehara, Wen Sun
Pessimistic Model-based Offline RL: PAC Bounds and Posterior Sampling under Partial Coverage. CoRR (2021)
Yichun Hu, Nathan Kallus, Masatoshi Uehara
Fast Rates for the Regret of Offline Reinforcement Learning. COLT (2021)
Chengchun Shi, Masatoshi Uehara, Nan Jiang
A Minimax Learning Approach to Off-Policy Evaluation in Partially Observable Markov Decision Processes. CoRR (2021)
Jonathan D. Chang, Masatoshi Uehara, Dhruv Sreenivas, Rahul Kidambi, Wen Sun
Mitigating Covariate Shift in Imitation Learning via Offline Data Without Great Coverage. CoRR (2021)
Jonathan D. Chang, Masatoshi Uehara, Dhruv Sreenivas, Rahul Kidambi, Wen Sun
Mitigating Covariate Shift in Imitation Learning via Offline Data With Partial Coverage. NeurIPS (2021)
Masatoshi Uehara, Masaaki Imaizumi, Nan Jiang, Nathan Kallus, Wen Sun, Tengyang Xie
Finite Sample Analysis of Minimax Offline Reinforcement Learning: Completeness, Fast Rates and First-Order Efficiency. CoRR (2021)
Masatoshi Uehara, Xuezhou Zhang, Wen Sun
Representation Learning for Online and Offline RL in Low-rank MDPs. CoRR (2021)
Nathan Kallus, Yuta Saito, Masatoshi Uehara
Optimal Off-Policy Evaluation from Multiple Logging Policies. ICML (2021)
Nathan Kallus, Masatoshi Uehara
Statistically Efficient Off-Policy Policy Gradients. ICML (2020)
Nathan Kallus, Masatoshi Uehara
Doubly Robust Off-Policy Value and Gradient Estimation for Deterministic Policies. NeurIPS (2020)
Masatoshi Uehara, Jiawei Huang, Nan Jiang
Minimax Weight and Q-Function Learning for Off-Policy Evaluation. ICML (2020)
Nathan Kallus, Masatoshi Uehara
Statistically Efficient Off-Policy Policy Gradients. CoRR (2020)
Nathan Kallus, Masatoshi Uehara
Efficient Evaluation of Natural Stochastic Policies in Offline Reinforcement Learning. CoRR (2020)
Nathan Kallus, Masatoshi Uehara
Double Reinforcement Learning for Efficient and Robust Off-Policy Evaluation. ICML (2020)
Masatoshi Uehara, Takafumi Kanamori, Takashi Takenouchi, Takeru Matsuda
A Unified Statistically Efficient Estimation Framework for Unnormalized Models. AISTATS (2020)
Nathan Kallus, Yuta Saito, Masatoshi Uehara
Optimal Off-Policy Evaluation from Multiple Logging Policies. CoRR (2020)
Masatoshi Uehara, Masahiro Kato, Shota Yasui
Off-Policy Evaluation and Learning for External Validity under a Covariate Shift. NeurIPS (2020)
Masahiro Kato, Masatoshi Uehara, Shota Yasui
Off-Policy Evaluation and Learning for External Validity under a Covariate Shift. CoRR (2020)
Nathan Kallus, Masatoshi Uehara
Double Reinforcement Learning for Efficient Off-Policy Evaluation in Markov Decision Processes. J. Mach. Learn. Res. 21 (2020)
Nathan Kallus, Masatoshi Uehara
Doubly Robust Off-Policy Value and Gradient Estimation for Deterministic Policies. CoRR (2020)
Masatoshi Uehara, Takeru Matsuda, Jae Kwang Kim
Imputation estimators for unnormalized models with missing data. AISTATS (2020)
Takeru Matsuda, Masatoshi Uehara, Aapo Hyvärinen
Information criteria for non-normalized models. CoRR (2019)
Nathan Kallus, Masatoshi Uehara
Efficiently Breaking the Curse of Horizon: Double Reinforcement Learning in Infinite-Horizon Processes. CoRR (2019)
Nathan Kallus, Masatoshi Uehara
Intrinsically Efficient, Stable, and Bounded Off-Policy Evaluation for Reinforcement Learning. CoRR (2019)
Nathan Kallus, Masatoshi Uehara
Double Reinforcement Learning for Efficient Off-Policy Evaluation in Markov Decision Processes. CoRR (2019)
Masatoshi Uehara, Takeru Matsuda, Jae Kwang Kim
Imputation estimators for unnormalized models with missing data. CoRR (2019)
Nathan Kallus, Masatoshi Uehara
Intrinsically Efficient, Stable, and Bounded Off-Policy Evaluation for Reinforcement Learning. NeurIPS (2019)
Masatoshi Uehara, Takafumi Kanamori, Takashi Takenouchi, Takeru Matsuda
Unified estimation framework for unnormalized models with statistical efficiency. CoRR (2019)
Masatoshi Uehara, Nan Jiang
Minimax Weight and Q-Function Learning for Off-Policy Evaluation. CoRR (2019)
Nathan Kallus, Xiaojie Mao, Masatoshi Uehara
Localized Debiased Machine Learning: Efficient Estimation of Quantile Treatment Effects, Conditional Value at Risk, and Beyond. CoRR (2019)
Masatoshi Uehara, Takeru Matsuda, Fumiyasu Komaki
Analysis of Noise Contrastive Estimation from the Perspective of Asymptotic Variance. CoRR (2018)