Dani Yogatama

Publication Activity (10 Years)

Years Active: 2009-2024
Publications (10 Years): 76

Top Topics

Decision Making Under Uncertainty

Text Generation

Structured Sparsity

Reinforcement Learning

Top Venues

Trans. Assoc. Comput. Linguistics

Publications

Aitor Ormazabal, Che Zheng, Cyprien de Masson d'Autume, Dani Yogatama, Deyu Fu, Donovan Ong, Eric Chen, Eugenie Lamprecht, Hai Pham, Isaac Ong, Kaloyan Aleksiev, Lei Li, Matthew Henderson, Max Bain, Mikel Artetxe, Nishant Relan, Piotr Padlewski, Qi Liu, Ren Chen, Samuel Phua, Yazheng Yang, Yi Tay, Yuqi Wang, Zhongkai Zhu, Zhihui Xie
Reka Core, Flash, and Edge: A Series of Powerful Multimodal Language Models. CoRR (2024)
Ting-Rui Chiang, Dani Yogatama
Understanding In-Context Learning with a Pelican Soup Framework. CoRR (2024)
Piotr Padlewski, Max Bain, Matthew Henderson, Zhongkai Zhu, Nishant Relan, Hai Pham, Donovan Ong, Kaloyan Aleksiev, Aitor Ormazabal, Samuel Phua, Ethan Yeo, Eugenie Lamprecht, Qi Liu, Yuqi Wang, Eric Chen, Deyu Fu, Lei Li, Che Zheng, Cyprien de Masson d'Autume, Dani Yogatama, Mikel Artetxe, Yi Tay
Vibe-Eval: A hard evaluation suite for measuring progress of multimodal language models. CoRR (2024)
Xianghao Kong, Ollie Liu, Han Li, Dani Yogatama, Greg Ver Steeg
Interpretable Diffusion via Information Decomposition. ICLR (2024)
Ollie Liu, Deqing Fu, Dani Yogatama, Willie Neiswanger
DeLLMa: A Framework for Decision Making Under Uncertainty with Large Language Models. CoRR (2024)
Deqing Fu, Ghazal Khalighinejad, Ollie Liu, Bhuwan Dhingra, Dani Yogatama, Robin Jia, Willie Neiswanger
IsoBench: Benchmarking Multimodal Foundation Models on Isomorphic Representations. CoRR (2024)
Ting-Rui Chiang, Dani Yogatama
The Distributional Hypothesis Does Not Fully Explain the Benefits of Masked Language Model Pretraining. CoRR (2023)
Ting-Rui Chiang, Xinyan Velocity Yu, Joshua Robinson, Ollie Liu, Isabelle Lee, Dani Yogatama
On Retrieval Augmentation and the Limitations of Language Model Training. CoRR (2023)
Devendra Singh Sachan, Mike Lewis, Dani Yogatama, Luke Zettlemoyer, Joelle Pineau, Manzil Zaheer
Questions Are All You Need to Train a Dense Passage Retriever. Trans. Assoc. Comput. Linguistics 11 (2023)
Xianghao Kong, Ollie Liu, Han Li, Dani Yogatama, Greg Ver Steeg
Interpretable Diffusion via Information Decomposition. CoRR (2023)
Yi Tay, Mostafa Dehghani, Samira Abnar, Hyung Won Chung, William Fedus, Jinfeng Rao, Sharan Narang, Vinh Q. Tran, Dani Yogatama, Donald Metzler
Scaling Laws vs Model Architectures: How does Inductive Bias Influence Scaling? EMNLP (Findings) (2023)
Paul Pu Liang, Yiwei Lyu, Xiang Fan, Jeffrey Tsaw, Yudong Liu, Shentong Mo, Dani Yogatama, Louis-Philippe Morency, Russ Salakhutdinov
High-Modality Multimodal Transformer: Quantifying Modality & Interaction Heterogeneity for High-Modality Representation Learning. Trans. Mach. Learn. Res. 2023 (2023)
Ting-Rui Chiang, Dani Yogatama
The Distributional Hypothesis Does Not Fully Explain the Benefits of Masked Language Model Pretraining. EMNLP (2023)
Yi Tay, Mostafa Dehghani, Samira Abnar, Hyung Won Chung, William Fedus, Jinfeng Rao, Sharan Narang, Vinh Q. Tran, Dani Yogatama, Donald Metzler
Scaling Laws vs Model Architectures: How does Inductive Bias Influence Scaling? CoRR (2022)
Yixuan Su, Tian Lan, Yahui Liu, Fangyu Liu, Dani Yogatama, Yan Wang, Lingpeng Kong, Nigel Collier
Language Models Can See: Plugging Visual Controls in Text Generation. CoRR (2022)
Qi Liu, Dani Yogatama, Phil Blunsom
Relational Memory-Augmented Language Models. Trans. Assoc. Comput. Linguistics 10 (2022)
Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, Ed H. Chi, Tatsunori Hashimoto, Oriol Vinyals, Percy Liang, Jeff Dean, William Fedus
Emergent Abilities of Large Language Models. Trans. Mach. Learn. Res. 2022 (2022)
Yixuan Su, Tian Lan, Yan Wang, Dani Yogatama, Lingpeng Kong, Nigel Collier
A Contrastive Framework for Neural Text Generation. NeurIPS (2022)
Paul Pu Liang, Yiwei Lyu, Xiang Fan, Shentong Mo, Dani Yogatama, Louis-Philippe Morency, Ruslan Salakhutdinov
HighMMT: Towards Modality and Task Generalization for High-Modality Representation Learning. CoRR (2022)
Qi Liu, Dani Yogatama, Phil Blunsom
Relational Memory Augmented Language Models. CoRR (2022)
Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, Ed H. Chi, Tatsunori Hashimoto, Oriol Vinyals, Percy Liang, Jeff Dean, William Fedus
Emergent Abilities of Large Language Models. CoRR (2022)
Yixuan Su, Tian Lan, Yan Wang, Dani Yogatama, Lingpeng Kong, Nigel Collier
A Contrastive Framework for Neural Text Generation. CoRR (2022)
Hao Peng, Jungo Kasai, Nikolaos Pappas, Dani Yogatama, Zhaofeng Wu, Lingpeng Kong, Roy Schwartz, Noah A. Smith
ABC: Attention with Bounded-memory Control. ACL (1) (2022)
Devendra Singh Sachan, Mike Lewis, Dani Yogatama, Luke Zettlemoyer, Joelle Pineau, Manzil Zaheer
Questions Are All You Need to Train a Dense Passage Retriever. CoRR (2022)
Yi Tay, Mostafa Dehghani, Jinfeng Rao, William Fedus, Samira Abnar, Hyung Won Chung, Sharan Narang, Dani Yogatama, Ashish Vaswani, Donald Metzler
Scale Efficiently: Insights from Pretraining and Finetuning Transformers. ICLR (2022)
Devendra Singh Sachan, Siva Reddy, William L. Hamilton, Chris Dyer, Dani Yogatama
End-to-End Training of Multi-Document Reader and Retriever for Open-Domain Question Answering. CoRR (2021)
Paul Michel, Sebastian Ruder, Dani Yogatama
Balancing Average and Worst-case Accuracy in Multitask Learning. CoRR (2021)
Hao Peng, Jungo Kasai, Nikolaos Pappas, Dani Yogatama, Zhaofeng Wu, Lingpeng Kong, Roy Schwartz, Noah A. Smith
ABC: Attention with Bounded-memory Control. CoRR (2021)
Stefan Daniel Dumitrescu, Petru Rebeja, Beáta Lorincz, Mihaela Gaman, Andrei-Marius Avram, Mihai Ilie, Andrei Pruteanu, Adriana Stan, Lorena Rosia, Cristina Iacobescu, Luciana Morogan, George Dima, Gabriel Marchidan, Traian Rebedea, Madalina Chitez, Dani Yogatama, Sebastian Ruder, Radu Tudor Ionescu, Razvan Pascanu, Viorica Patraucean
LiRo: Benchmark and leaderboard for Romanian language tasks. NeurIPS Datasets and Benchmarks (2021)
Dani Yogatama, Cyprien de Masson d'Autume, Lingpeng Kong
Adaptive Semiparametric Language Models. CoRR (2021)
Devendra Singh Sachan, Siva Reddy, William L. Hamilton, Chris Dyer, Dani Yogatama
End-to-End Training of Multi-Document Reader and Retriever for Open-Domain Question Answering. NeurIPS (2021)
Angeliki Lazaridou, Adhiguna Kuncoro, Elena Gribovskaya, Devang Agrawal, Adam Liska, Tayfun Terzi, Mai Gimenez, Cyprien de Masson d'Autume, Sebastian Ruder, Dani Yogatama, Kris Cao, Tomás Kociský, Susannah Young, Phil Blunsom
Pitfalls of Static Language Modelling. CoRR (2021)
Yi Tay, Mostafa Dehghani, Jinfeng Rao, William Fedus, Samira Abnar, Hyung Won Chung, Sharan Narang, Dani Yogatama, Ashish Vaswani, Donald Metzler
Scale Efficiently: Insights from Pre-training and Fine-tuning Transformers. CoRR (2021)
Hao Peng, Nikolaos Pappas, Dani Yogatama, Roy Schwartz, Noah A. Smith, Lingpeng Kong
Random Feature Attention. CoRR (2021)
Dani Yogatama, Cyprien de Masson d'Autume, Lingpeng Kong
Adaptive Semiparametric Language Models. Trans. Assoc. Comput. Linguistics 9 (2021)
Hao Peng, Nikolaos Pappas, Dani Yogatama, Roy Schwartz, Noah A. Smith, Lingpeng Kong
Random Feature Attention. ICLR (2021)
Jungo Kasai, Hao Peng, Yizhe Zhang, Dani Yogatama, Gabriel Ilharco, Nikolaos Pappas, Yi Mao, Weizhu Chen, Noah A. Smith
Finetuning Pretrained Transformers into RNNs. EMNLP (1) (2021)
Jungo Kasai, Hao Peng, Yizhe Zhang, Dani Yogatama, Gabriel Ilharco, Nikolaos Pappas, Yi Mao, Weizhu Chen, Noah A. Smith
Finetuning Pretrained Transformers into RNNs. CoRR (2021)
Angeliki Lazaridou, Adhiguna Kuncoro, Elena Gribovskaya, Devang Agrawal, Adam Liska, Tayfun Terzi, Mai Gimenez, Cyprien de Masson d'Autume, Tomás Kociský, Sebastian Ruder, Dani Yogatama, Kris Cao, Susannah Young, Phil Blunsom
Mind the Gap: Assessing Temporal Generalization in Neural Language Models. NeurIPS (2021)
Mikel Artetxe, Sebastian Ruder, Dani Yogatama, Gorka Labaka, Eneko Agirre
A Call for More Rigor in Unsupervised Cross-lingual Learning. ACL (2020)
Kris Cao, Dani Yogatama
Modelling Latent Skills for Multitask Language Generation. CoRR (2020)
Mikel Artetxe, Sebastian Ruder, Dani Yogatama, Gorka Labaka, Eneko Agirre
A Call for More Rigor in Unsupervised Cross-lingual Learning. CoRR (2020)
Adhiguna Kuncoro, Lingpeng Kong, Daniel Fried, Dani Yogatama, Laura Rimell, Chris Dyer, Phil Blunsom
Syntactic Structure Distillation Pretraining for Bidirectional Encoders. Trans. Assoc. Comput. Linguistics 8 (2020)
Mikel Artetxe, Sebastian Ruder, Dani Yogatama
On the Cross-lingual Transferability of Monolingual Representations. ACL (2020)
Po-Sen Huang, Huan Zhang, Ray Jiang, Robert Stanforth, Johannes Welbl, Jack Rae, Vishal Maini, Dani Yogatama, Pushmeet Kohli
Reducing Sentiment Bias in Language Models via Counterfactual Evaluation. EMNLP (Findings) (2020)
Adhiguna Kuncoro, Lingpeng Kong, Daniel Fried, Dani Yogatama, Laura Rimell, Chris Dyer, Phil Blunsom
Syntactic Structure Distillation Pretraining For Bidirectional Encoders. CoRR (2020)
Lingpeng Kong, Cyprien de Masson d'Autume, Lei Yu, Wang Ling, Zihang Dai, Dani Yogatama
A Mutual Information Maximization Perspective of Language Representation Learning. ICLR (2020)
Lingpeng Kong, Gábor Melis, Wang Ling, Lei Yu, Dani Yogatama
Variational Smoothing in Recurrent Neural Network Language Models. ICLR (Poster) (2019)
Po-Sen Huang, Robert Stanforth, Johannes Welbl, Chris Dyer, Dani Yogatama, Sven Gowal, Krishnamurthy Dvijotham, Pushmeet Kohli
Achieving Verified Robustness to Symbol Substitutions via Interval Bound Propagation. EMNLP/IJCNLP (1) (2019)
Po-Sen Huang, Robert Stanforth, Johannes Welbl, Chris Dyer, Dani Yogatama, Sven Gowal, Krishnamurthy Dvijotham, Pushmeet Kohli
Achieving Verified Robustness to Symbol Substitutions via Interval Bound Propagation. CoRR (2019)
Po-Sen Huang, Huan Zhang, Ray Jiang, Robert Stanforth, Johannes Welbl, Jack Rae, Vishal Maini, Dani Yogatama, Pushmeet Kohli
Reducing Sentiment Bias in Language Models via Counterfactual Evaluation. CoRR (2019)
Cyprien de Masson d'Autume, Sebastian Ruder, Lingpeng Kong, Dani Yogatama
Episodic Memory in Lifelong Language Learning. NeurIPS (2019)
Mikel Artetxe, Sebastian Ruder, Dani Yogatama
On the Cross-lingual Transferability of Monolingual Representations. CoRR (2019)
Cyprien de Masson d'Autume, Sebastian Ruder, Lingpeng Kong, Dani Yogatama
Episodic Memory in Lifelong Language Learning. CoRR (2019)
Lingpeng Kong, Gábor Melis, Wang Ling, Lei Yu, Dani Yogatama
Variational Smoothing in Recurrent Neural Network Language Models. CoRR (2019)
Jean Maillard, Stephen Clark, Dani Yogatama
Jointly learning sentence embeddings and syntax with unsupervised Tree-LSTMs. Nat. Lang. Eng. 25 (4) (2019)
Dani Yogatama, Cyprien de Masson d'Autume, Jerome T. Connor, Tomás Kociský, Mike Chrzanowski, Lingpeng Kong, Angeliki Lazaridou, Wang Ling, Lei Yu, Chris Dyer, Phil Blunsom
Learning and Evaluating General Linguistic Intelligence. CoRR (2019)
Lingpeng Kong, Cyprien de Masson d'Autume, Wang Ling, Lei Yu, Zihang Dai, Dani Yogatama
A Mutual Information Maximization Perspective of Language Representation Learning. CoRR (2019)
Oriol Vinyals, Igor Babuschkin, Wojciech M. Czarnecki, Michaël Mathieu, Andrew Dudzik, Junyoung Chung, David H. Choi, Richard Powell, Timo Ewalds, Petko Georgiev, Junhyuk Oh, Dan Horgan, Manuel Kroiss, Ivo Danihelka, Aja Huang, Laurent Sifre, Trevor Cai, John P. Agapiou, Max Jaderberg, Alexander Sasha Vezhnevets, Rémi Leblond, Tobias Pohlen, Valentin Dalibard, David Budden, Yury Sulsky, James Molloy, Tom Le Paine, Çaglar Gülçehre, Ziyu Wang, Tobias Pfaff, Yuhuai Wu, Roman Ring, Dani Yogatama, Dario Wünsch, Katrina McKinney, Oliver Smith, Tom Schaul, Timothy P. Lillicrap, Koray Kavukcuoglu, Demis Hassabis, Chris Apps, David Silver
Grandmaster level in StarCraft II using multi-agent reinforcement learning. Nat. 575 (7782) (2019)
Dani Yogatama, Yishu Miao, Gábor Melis, Wang Ling, Adhiguna Kuncoro, Chris Dyer, Phil Blunsom
Memory Architectures in Recurrent Neural Network Language Models. ICLR (Poster) (2018)
Adhiguna Kuncoro, Chris Dyer, John Hale, Dani Yogatama, Stephen Clark, Phil Blunsom
LSTMs Can Learn Syntax-Sensitive Dependencies Well, But Modeling Structure Makes Them Better. ACL (1) (2018)
Dani Yogatama, Phil Blunsom, Chris Dyer, Edward Grefenstette, Wang Ling
Learning to Compose Words into Sentences with Reinforcement Learning. ICLR (Poster) (2017)
Wang Ling, Dani Yogatama, Chris Dyer, Phil Blunsom
Program Induction by Rationale Generation: Learning to Solve and Explain Algebraic Word Problems. CoRR (2017)
Wang Ling, Dani Yogatama, Chris Dyer, Phil Blunsom
Program Induction by Rationale Generation: Learning to Solve and Explain Algebraic Word Problems. ACL (1) (2017)
Jean Maillard, Stephen Clark, Dani Yogatama
Jointly Learning Sentence Embeddings and Syntax with Unsupervised Tree-LSTMs. CoRR (2017)
Dani Yogatama, Chris Dyer, Wang Ling, Phil Blunsom
Generative and Discriminative Text Classification with Recurrent Neural Networks. CoRR (2017)
Dani Yogatama, Phil Blunsom, Chris Dyer, Edward Grefenstette, Wang Ling
Learning to Compose Words into Sentences with Reinforcement Learning. CoRR (2016)
Dario Amodei, Sundaram Ananthanarayanan, Rishita Anubhai, Jingliang Bai, Eric Battenberg, Carl Case, Jared Casper, Bryan Catanzaro, Jingdong Chen, Mike Chrzanowski, Adam Coates, Greg Diamos, Erich Elsen, Jesse H. Engel, Linxi Fan, Christopher Fougner, Awni Y. Hannun, Billy Jun, Tony Han, Patrick LeGresley, Xiangang Li, Libby Lin, Sharan Narang, Andrew Y. Ng, Sherjil Ozair, Ryan Prenger, Sheng Qian, Jonathan Raiman, Sanjeev Satheesh, David Seetapun, Shubho Sengupta, Chong Wang, Yi Wang, Zhiqian Wang, Bo Xiao, Yan Xie, Dani Yogatama, Jun Zhan, Zhenyao Zhu
Deep Speech 2 : End-to-End Speech Recognition in English and Mandarin. ICML (2016)
Dani Yogatama, Lingpeng Kong, Noah A. Smith
Bayesian Optimization of Text Representations. EMNLP (2015)
Manaal Faruqui, Yulia Tsvetkov, Dani Yogatama, Chris Dyer, Noah A. Smith
Sparse Overcomplete Word Vector Representations. CoRR (2015)
Dani Yogatama, Daniel Gillick, Nevena Lazic
Embedding Methods for Fine Grained Entity Type Classification. ACL (2) (2015)
Dani Yogatama, Manaal Faruqui, Chris Dyer, Noah A. Smith
Learning Word Representations with Hierarchical Sparse Coding. ICML (2015)
Dani Yogatama, Noah A. Smith
Bayesian Optimization of Text Representations. CoRR (2015)
Dario Amodei, Rishita Anubhai, Eric Battenberg, Carl Case, Jared Casper, Bryan Catanzaro, Jingdong Chen, Mike Chrzanowski, Adam Coates, Greg Diamos, Erich Elsen, Jesse H. Engel, Linxi Fan, Christopher Fougner, Tony Han, Awni Y. Hannun, Billy Jun, Patrick LeGresley, Libby Lin, Sharan Narang, Andrew Y. Ng, Sherjil Ozair, Ryan Prenger, Jonathan Raiman, Sanjeev Satheesh, David Seetapun, Shubho Sengupta, Yi Wang, Zhiqian Wang, Chong Wang, Bo Xiao, Dani Yogatama, Jun Zhan, Zhenyao Zhu
Deep Speech 2: End-to-End Speech Recognition in English and Mandarin. CoRR (2015)
Dani Yogatama, Fei Liu, Noah A. Smith
Extractive Summarization by Maximizing Semantic Volume. EMNLP (2015)
Manaal Faruqui, Yulia Tsvetkov, Dani Yogatama, Chris Dyer, Noah A. Smith
Sparse Overcomplete Word Vector Representations. ACL (1) (2015)
Dani Yogatama, Noah A. Smith
Linguistic Structured Sparsity in Text Categorization. ACL (1) (2014)
Dani Yogatama, Chong Wang, Bryan R. Routledge, Noah A. Smith, Eric P. Xing
Dynamic Language Models for Streaming Text. Trans. Assoc. Comput. Linguistics 2 (2014)
Dani Yogatama, Noah A. Smith
Making the Most of Bag of Words: Sentence Regularization with Alternating Direction Method of Multipliers. ICML (2014)
Dani Yogatama, Manaal Faruqui, Chris Dyer, Noah A. Smith
Learning Word Representations with Hierarchical Sparse Coding. CoRR (2014)
Dani Yogatama, Gideon Mann
Efficient Transfer Learning Method for Automatic Hyperparameter Tuning. AISTATS (2014)
Dani Yogatama, Bryan R. Routledge, Noah A. Smith
A Sparse and Adaptive Prior for Time-Dependent Model Parameters. CoRR (2013)
Tae Yano, Dani Yogatama, Noah A. Smith
A Penny for Your Tweets: Campaign Contributions and Capitol Hill Microblogs. ICWSM (2013)
Dani Yogatama, Yanchuan Sim, Noah A. Smith
A Probabilistic Model for Canonicalizing Named Entity Mentions. ACL (1) (2012)
Dani Yogatama, Michael Heilman, Brendan O'Connor, Chris Dyer, Bryan R. Routledge, Noah A. Smith
Predicting a Scientific Community's Response to an Article. EMNLP (2011)
Kevin Gimpel, Nathan Schneider, Brendan O'Connor, Dipanjan Das, Daniel Mills, Jacob Eisenstein, Michael Heilman, Dani Yogatama, Jeffrey Flanigan, Noah A. Smith
Part-of-Speech Tagging for Twitter: Annotation, Features, and Experiments. ACL (2) (2011)
Dani Yogatama, Kumiko Tanaka-Ishii
Multilingual Spectral Clustering Using Document Similarity Propagation. EMNLP (2009)