Iz Beltagy

Publication Activity (10 Years)

Years Active: 2018-2024
Publications (10 Years): 69

Top Topics

Document Summarization

Information Extraction

Natural Language Processing

Top Venues

EMNLP (Findings)

Publications

Dirk Groeneveld, Iz Beltagy, Pete Walsh, Akshita Bhagia, Rodney Kinney, Oyvind Tafjord, Ananya Harsh Jha, Hamish Ivison, Ian Magnusson, Yizhong Wang, Shane Arora, David Atkinson, Russell Authur, Khyathi Raghavi Chandu, Arman Cohan, Jennifer Dumas, Yanai Elazar, Yuling Gu, Jack Hessel, Tushar Khot, William Merrill, Jacob Morrison, Niklas Muennighoff, Aakanksha Naik, Crystal Nam, Matthew E. Peters, Valentina Pyatkin, Abhilasha Ravichander, Dustin Schwenk, Saurabh Shah, Will Smith, Emma Strubell, Nishant Subramani, Mitchell Wortsman, Pradeep Dasigi, Nathan Lambert, Kyle Richardson, Luke Zettlemoyer, Jesse Dodge, Kyle Lo, Luca Soldaini, Noah A. Smith, Hannaneh Hajishirzi
OLMo: Accelerating the Science of Language Models. CoRR (2024)
Muhammad Khalifa, David Wadden, Emma Strubell, Honglak Lee, Lu Wang, Iz Beltagy, Hao Peng
Source-Aware Training Enables Knowledge Attribution in Language Models. CoRR (2024)
Rabeeh Karimi Mahabadi, Hamish Ivison, Jaesung Tae, James Henderson, Iz Beltagy, Matthew E. Peters, Arman Cohan
TESS: Text-to-Text Self-Conditioned Simplex Diffusion. EACL (1) (2024)
Luca Soldaini, Rodney Kinney, Akshita Bhagia, Dustin Schwenk, David Atkinson, Russell Authur, Ben Bogin, Khyathi Chandu, Jennifer Dumas, Yanai Elazar, Valentin Hofmann, Ananya Harsh Jha, Sachin Kumar, Li Lucy, Xinxi Lyu, Nathan Lambert, Ian Magnusson, Jacob Morrison, Niklas Muennighoff, Aakanksha Naik, Crystal Nam, Matthew E. Peters, Abhilasha Ravichander, Kyle Richardson, Zejiang Shen, Emma Strubell, Nishant Subramani, Oyvind Tafjord, Pete Walsh, Luke Zettlemoyer, Noah A. Smith, Hannaneh Hajishirzi, Iz Beltagy, Dirk Groeneveld, Jesse Dodge, Kyle Lo
Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Research. CoRR (2024)
Rodney Kinney, Chloe Anastasiades, Russell Authur, Iz Beltagy, Jonathan Bragg, Alexandra Buraczynski, Isabel Cachola, Stefan Candra, Yoganand Chandrasekhar, Arman Cohan, Miles Crawford, Doug Downey, Jason Dunkelberger, Oren Etzioni, Rob Evans, Sergey Feldman, Joseph Gorney, David Graham, Fangzhou Hu, Regan Huff, Daniel King, Sebastian Kohlmeier, Bailey Kuehl, Michael Langan, Daniel Lin, Haokun Liu, Kyle Lo, Jaron Lochner, Kelsey MacMillan, Tyler Murray, Chris Newell, Smita Rao, Shaurya Rohatgi, Paul Sayre, Zejiang Shen, Amanpreet Singh, Luca Soldaini, Shivashankar Subramanian, Amber Tanaka, Alex D. Wade, Linda Wagner, Lucy Lu Wang, Chris Wilhelm, Caroline Wu, Jiangjiang Yang, Angele Zamarron, Madeleine van Zuylen, Daniel S. Weld
The Semantic Scholar Open Data Platform. CoRR (2023)
Dirk Groeneveld, Anas Awadalla, Iz Beltagy, Akshita Bhagia, Ian Magnusson, Hao Peng, Oyvind Tafjord, Pete Walsh, Kyle Richardson, Jesse Dodge
Catwalk: A Unified Language Model Evaluation Framework for Many Datasets. CoRR (2023)
Ian Magnusson, Akshita Bhagia, Valentin Hofmann, Luca Soldaini, Ananya Harsh Jha, Oyvind Tafjord, Dustin Schwenk, Evan Pete Walsh, Yanai Elazar, Kyle Lo, Dirk Groeneveld, Iz Beltagy, Hannaneh Hajishirzi, Noah A. Smith, Kyle Richardson, Jesse Dodge
Paloma: A Benchmark for Evaluating Language Model Fit. CoRR (2023)
Zhaofeng Wu, William Merrill, Hao Peng, Iz Beltagy, Noah A. Smith
Transparency Helps Reveal When Language Models Learn Meaning. Trans. Assoc. Comput. Linguistics 11 (2023)
Hao Peng, Qingqing Cao, Jesse Dodge, Matthew E. Peters, Jared Fernandez, Tom Sherborne, Kyle Lo, Sam Skjonsberg, Emma Strubell, Darrell Plessas, Iz Beltagy, Evan Pete Walsh, Noah A. Smith, Hannaneh Hajishirzi
Efficiency Pentathlon: A Standardized Arena for Efficiency Evaluation. CoRR (2023)
Rabeeh Karimi Mahabadi, Jaesung Tae, Hamish Ivison, James Henderson, Iz Beltagy, Matthew E. Peters, Arman Cohan
TESS: Text-to-Text Self-Conditioned Simplex Diffusion. CoRR (2023)
Yizhong Wang, Hamish Ivison, Pradeep Dasigi, Jack Hessel, Tushar Khot, Khyathi Chandu, David Wadden, Kelsey MacMillan, Noah A. Smith, Iz Beltagy, Hannaneh Hajishirzi
How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resources. NeurIPS (2023)
Hamish Ivison, Yizhong Wang, Valentina Pyatkin, Nathan Lambert, Matthew Peters, Pradeep Dasigi, Joel Jang, David Wadden, Noah A. Smith, Iz Beltagy, Hannaneh Hajishirzi
Camels in a Changing Climate: Enhancing LM Adaptation with Tulu 2. CoRR (2023)
Ananya Harsh Jha, Dirk Groeneveld, Emma Strubell, Iz Beltagy
Large Language Model Distillation Doesn't Need a Teacher. CoRR (2023)
Yizhong Wang, Hamish Ivison, Pradeep Dasigi, Jack Hessel, Tushar Khot, Khyathi Raghavi Chandu, David Wadden, Kelsey MacMillan, Noah A. Smith, Iz Beltagy, Hannaneh Hajishirzi
How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resources. CoRR (2023)
Qinyuan Ye, Iz Beltagy, Matthew Peters, Xiang Ren, Hannaneh Hajishirzi
FiD-ICL: A Fusion-in-Decoder Approach for Efficient In-Context Learning. ACL (1) (2023)
Xinxi Lyu, Sewon Min, Iz Beltagy, Luke Zettlemoyer, Hannaneh Hajishirzi
Z-ICL: Zero-Shot In-Context Learning with Pseudo-Demonstrations. ACL (1) (2023)
Zhaofeng Wu, William Merrill, Hao Peng, Iz Beltagy, Noah A. Smith
Transparency Helps Reveal When Language Models Learn Meaning. CoRR (2022)
Michael J. Cafarella, Michael R. Anderson, Iz Beltagy, Arie Cattan, Sarah E. Chasins, Ido Dagan, Doug Downey, Oren Etzioni, Sergey Feldman, Tian Gao, Tom Hope, Kexin Huang, Sophie Johnson, Daniel King, Kyle Lo, Yuze Lou, Matthew D. Shapiro, Dinghao Shen, Shivashankar Subramanian, Lucy Lu Wang, Yuning Wang, Yitong Wang, Daniel S. Weld, Jenny M. Vo-Phamhi, Anna Zeng, Jiayun Zou
Infrastructure for Rapid Open Knowledge Network Development. AI Mag. 43 (1) (2022)
Teven Le Scao, Thomas Wang, Daniel Hesslow, Lucile Saulnier, Stas Bekman, M. Saiful Bari, Stella Biderman, Hady Elsahar, Niklas Muennighoff, Jason Phang, Ofir Press, Colin Raffel, Victor Sanh, Sheng Shen, Lintang Sutawika, Jaesung Tae, Zheng Xin Yong, Julien Launay, Iz Beltagy
What Language Model to Train if You Have One Million GPU Hours? CoRR (2022)
Sheng Shen, Pete Walsh, Kurt Keutzer, Jesse Dodge, Matthew E. Peters, Iz Beltagy
Staged Training for Transformer Language Models. ICML (2022)
Thomas Wang, Adam Roberts, Daniel Hesslow, Teven Le Scao, Hyung Won Chung, Iz Beltagy, Julien Launay, Colin Raffel
What Language Model Architecture and Pretraining Objective Works Best for Zero-Shot Generalization? ICML (2022)
Zhaofeng Wu, Robert L. Logan IV, Pete Walsh, Akshita Bhagia, Dirk Groeneveld, Sameer Singh, Iz Beltagy
Continued Pretraining for Better Zero- and Few-Shot Promptability. CoRR (2022)
David Wadden, Kyle Lo, Bailey Kuehl, Arman Cohan, Iz Beltagy, Lucy Lu Wang, Hannaneh Hajishirzi
SciFact-Open: Towards open-domain scientific claim verification. CoRR (2022)
David Wadden, Kyle Lo, Lucy Lu Wang, Arman Cohan, Iz Beltagy, Hannaneh Hajishirzi
MultiVerS: Improving scientific claim verification with weak supervision and full-document context. NAACL-HLT (Findings) (2022)
Ana Marasovic, Iz Beltagy, Doug Downey, Matthew E. Peters
Few-Shot Self-Rationalization with Natural Language Prompts. NAACL-HLT (Findings) (2022)
Wen Xiao, Iz Beltagy, Giuseppe Carenini, Arman Cohan
PRIMERA: Pyramid-based Masked Sentence Pre-training for Multi-document Summarization. ACL (1) (2022)
Teven Le Scao, Angela Fan, Christopher Akiki, Ellie Pavlick, Suzana Ilic, Daniel Hesslow, Roman Castagné, Alexandra Sasha Luccioni, François Yvon, Matthias Gallé, Jonathan Tow, Alexander M. Rush, Stella Biderman, Albert Webson, Pawan Sasanka Ammanamanchi, Thomas Wang, Benoît Sagot, Niklas Muennighoff, Albert Villanova del Moral, Olatunji Ruwase, Rachel Bawden, Stas Bekman, Angelina McMillan-Major, Iz Beltagy, Huu Nguyen, Lucile Saulnier, Samson Tan, Pedro Ortiz Suarez, Victor Sanh, Hugo Laurençon, Yacine Jernite, Julien Launay, Margaret Mitchell, Colin Raffel, Aaron Gokaslan, Adi Simhi, Aitor Soroa, Alham Fikri Aji, Amit Alfassy, Anna Rogers, Ariel Kreisberg Nitzav, Canwen Xu, Chenghao Mou, Chris Emezue, Christopher Klamm, Colin Leong, Daniel van Strien, David Ifeoluwa Adelani, et al.
BLOOM: A 176B-Parameter Open-Access Multilingual Language Model. CoRR (2022)
David Wadden, Kyle Lo, Bailey Kuehl, Arman Cohan, Iz Beltagy, Lucy Lu Wang, Hannaneh Hajishirzi
SciFact-Open: Towards open-domain scientific claim verification. EMNLP (Findings) (2022)
Iz Beltagy, Arman Cohan, Robert L. Logan IV, Sewon Min, Sameer Singh Chauhan
Zero- and Few-Shot NLP with Pretrained Language Models. ACL (tutorial) (2022)
Sheng Shen, Pete Walsh, Kurt Keutzer, Jesse Dodge, Matthew E. Peters, Iz Beltagy
Staged Training for Transformer Language Models. CoRR (2022)
Zhaofeng Wu, Robert L. Logan IV, Pete Walsh, Akshita Bhagia, Dirk Groeneveld, Sameer Singh, Iz Beltagy
Continued Pretraining for Better Zero- and Few-Shot Promptability. EMNLP (2022)
Teven Le Scao, Thomas Wang, Daniel Hesslow, Stas Bekman, M. Saiful Bari, Stella Biderman, Hady Elsahar, Niklas Muennighoff, Jason Phang, Ofir Press, Colin Raffel, Victor Sanh, Sheng Shen, Lintang Sutawika, Jaesung Tae, Zheng Xin Yong, Julien Launay, Iz Beltagy
What Language Model to Train if You Have One Million GPU Hours? EMNLP (Findings) (2022)
Thomas Wang, Adam Roberts, Daniel Hesslow, Teven Le Scao, Hyung Won Chung, Iz Beltagy, Julien Launay, Colin Raffel
What Language Model Architecture and Pretraining Objective Work Best for Zero-Shot Generalization? CoRR (2022)
Daniel King, Zejiang Shen, Nishant Subramani, Daniel S. Weld, Iz Beltagy, Doug Downey
Don't Say What You Don't Know: Improving the Consistency of Abstractive Summarization by Constraining Beam Search. CoRR (2022)
Xinxi Lyu, Sewon Min, Iz Beltagy, Luke Zettlemoyer, Hannaneh Hajishirzi
Z-ICL: Zero-Shot In-Context Learning with Pseudo-Demonstrations. CoRR (2022)
Rahul Nadkarni, David Wadden, Iz Beltagy, Noah A. Smith, Hannaneh Hajishirzi, Tom Hope
Scientific Language Models for Biomedical Knowledge Base Completion: An Empirical Study. AKBC (2021)
Jay DeYoung, Iz Beltagy, Madeleine van Zuylen, Bailey Kuehl, Lucy Lu Wang
MS2: Multi-Document Summarization of Medical Studies. CoRR (2021)
Jonathan Bragg, Arman Cohan, Kyle Lo, Iz Beltagy
FLEX: Unifying Evaluation for Few-Shot NLP. NeurIPS (2021)
Pradeep Dasigi, Kyle Lo, Iz Beltagy, Arman Cohan, Noah A. Smith, Matt Gardner
A Dataset of Information-Seeking Questions and Answers Anchored in Research Papers. NAACL-HLT (2021)
Wen Xiao, Iz Beltagy, Giuseppe Carenini, Arman Cohan
PRIMER: Pyramid-based Masked Sentence Pre-training for Multi-document Summarization. CoRR (2021)
Arie Cattan, Sophie Johnson, Daniel S. Weld, Ido Dagan, Iz Beltagy, Doug Downey, Tom Hope
SciCo: Hierarchical Cross-Document Coreference for Scientific Concepts. AKBC (2021)
Arie Cattan, Sophie Johnson, Daniel S. Weld, Ido Dagan, Iz Beltagy, Doug Downey, Tom Hope
SciCo: Hierarchical Cross-Document Coreference for Scientific Concepts. CoRR (2021)
Rahul Nadkarni, David Wadden, Iz Beltagy, Noah A. Smith, Hannaneh Hajishirzi, Tom Hope
Scientific Language Models for Biomedical Knowledge Base Completion: An Empirical Study. CoRR (2021)
Ana Marasovic, Iz Beltagy, Doug Downey, Matthew E. Peters
Few-Shot Self-Rationalization with Natural Language Prompts. CoRR (2021)
Pradeep Dasigi, Kyle Lo, Iz Beltagy, Arman Cohan, Noah A. Smith, Matt Gardner
A Dataset of Information-Seeking Questions and Answers Anchored in Research Papers. CoRR (2021)
Jonathan Bragg, Arman Cohan, Kyle Lo, Iz Beltagy
FLEX: Unifying Evaluation for Few-Shot NLP. CoRR (2021)
Jay DeYoung, Iz Beltagy, Madeleine van Zuylen, Bailey Kuehl, Lucy Lu Wang
MS\^2: Multi-Document Summarization of Medical Studies. EMNLP (1) (2021)
Avi Caciularu, Arman Cohan, Iz Beltagy, Matthew E. Peters, Arie Cattan, Ido Dagan
CDLM: Cross-Document Language Modeling. EMNLP (Findings) (2021)
David Wadden, Kyle Lo, Lucy Lu Wang, Arman Cohan, Iz Beltagy, Hannaneh Hajishirzi
LongChecker: Improving scientific claim verification by modeling full-abstract context. CoRR (2021)
Avi Caciularu, Arman Cohan, Iz Beltagy, Matthew E. Peters, Arie Cattan, Ido Dagan
Cross-Document Language Modeling. CoRR (2021)
Iz Beltagy, Matthew E. Peters, Arman Cohan
Longformer: The Long-Document Transformer. CoRR (2020)
Sarthak Jain, Madeleine van Zuylen, Hannaneh Hajishirzi, Iz Beltagy
SciREX: A Challenge Dataset for Document-Level Information Extraction. ACL (2020)
Suchin Gururangan, Ana Marasovic, Swabha Swayamdipta, Kyle Lo, Iz Beltagy, Doug Downey, Noah A. Smith
Don't Stop Pretraining: Adapt Language Models to Domains and Tasks. CoRR (2020)
Sarthak Jain, Madeleine van Zuylen, Hannaneh Hajishirzi, Iz Beltagy
SciREX: A Challenge Dataset for Document-Level Information Extraction. CoRR (2020)
Suchin Gururangan, Ana Marasovic, Swabha Swayamdipta, Kyle Lo, Iz Beltagy, Doug Downey, Noah A. Smith
Don't Stop Pretraining: Adapt Language Models to Domains and Tasks. ACL (2020)
Arman Cohan, Sergey Feldman, Iz Beltagy, Doug Downey, Daniel S. Weld
SPECTER: Document-level Representation Learning using Citation-informed Transformers. ACL (2020)
Arman Cohan, Sergey Feldman, Iz Beltagy, Doug Downey, Daniel S. Weld
SPECTER: Document-level Representation Learning using Citation-informed Transformers. CoRR (2020)
Iz Beltagy, Kyle Lo, Waleed Ammar
Combining Distant and Direct Supervision for Neural Relation Extraction. NAACL-HLT (1) (2019)
Iz Beltagy, Kyle Lo, Arman Cohan
SciBERT: A Pretrained Language Model for Scientific Text. EMNLP/IJCNLP (1) (2019)
Arman Cohan, Iz Beltagy, Daniel King, Bhavana Dalvi, Daniel S. Weld
Pretrained Language Models for Sequential Sentence Classification. EMNLP/IJCNLP (1) (2019)
Arman Cohan, Iz Beltagy, Daniel King, Bhavana Dalvi, Daniel S. Weld
Pretrained Language Models for Sequential Sentence Classification. CoRR (2019)
Iz Beltagy, Arman Cohan, Kyle Lo
SciBERT: Pretrained Contextualized Embeddings for Scientific Text. CoRR (2019)
Mark Neumann, Daniel King, Iz Beltagy, Waleed Ammar
ScispaCy: Fast and Robust Models for Biomedical Natural Language Processing. CoRR (2019)
Mark Neumann, Daniel King, Iz Beltagy, Waleed Ammar
ScispaCy: Fast and Robust Models for Biomedical Natural Language Processing. BioNLP@ACL (2019)
Waleed Ammar, Dirk Groeneveld, Chandra Bhagavatula, Iz Beltagy, Miles Crawford, Doug Downey, Jason Dunkelberger, Ahmed Elgohary, Sergey Feldman, Vu Ha, Rodney Kinney, Sebastian Kohlmeier, Kyle Lo, Tyler Murray, Hsu-Han Ooi, Matthew E. Peters, Joanna Power, Sam Skjonsberg, Lucy Lu Wang, Chris Wilhelm, Zheng Yuan, Madeleine van Zuylen, Oren Etzioni
Construction of the Literature Graph in Semantic Scholar. NAACL-HLT (3) (2018)
Ahmed Alkhateeb, Iz Beltagy
Machine Learning for Reliable mmWave Systems: Blockage Prediction and Proactive Handoff. CoRR (2018)
Ahmed Alkhateeb, Iz Beltagy, Sam Alex
Machine Learning for Reliable MMWAVE Systems: Blockage Prediction and Proactive Handoff. GlobalSIP (2018)
Iz Beltagy, Kyle Lo, Waleed Ammar
Improving Distant Supervision with Maxpooled Attention and Sentence-Level Supervision. CoRR (2018)
Waleed Ammar, Dirk Groeneveld, Chandra Bhagavatula, Iz Beltagy, Miles Crawford, Doug Downey, Jason Dunkelberger, Ahmed Elgohary, Sergey Feldman, Vu Ha, Rodney Kinney, Sebastian Kohlmeier, Kyle Lo, Tyler Murray, Hsu-Han Ooi, Matthew E. Peters, Joanna Power, Sam Skjonsberg, Lucy Lu Wang, Chris Wilhelm, Zheng Yuan, Madeleine van Zuylen, Oren Etzioni
Construction of the Literature Graph in Semantic Scholar. CoRR (2018)