NeurIPS Datasets and Benchmarks

Publications

2021

Joseph Suarez, Yilun Du, Clare Zhu, Igor Mordatch, Phillip Isola
The Neural MMO Platform for Massively Multiagent Research. NeurIPS Datasets and Benchmarks (2021)
Francesco Croce, Maksym Andriushchenko, Vikash Sehwag, Edoardo Debenedetti, Nicolas Flammarion, Mung Chiang, Prateek Mittal, Matthias Hein
RobustBench: a standardized adversarial robustness benchmark. NeurIPS Datasets and Benchmarks (2021)
Tudor Mare, Georgian-Emilian Duta, Mariana-Iuliana Georgescu, Adrian Sandru, Bogdan Alexe, Marius Popescu, Radu Tudor Ionescu
A realistic approach to generate masked faces applied on two novel masked face recognition data sets. NeurIPS Datasets and Benchmarks (2021)
Minghui Chen, Zhiqiang Wang, Feng Zheng
Benchmarks for Corruption Invariant Person Re-identification. NeurIPS Datasets and Benchmarks (2021)
Bernd Bischl, Giuseppe Casalicchio, Matthias Feurer, Pieter Gijsbers, Frank Hutter, Michel Lang, Rafael Gomes Mantovani, Jan N. van Rijn, Joaquin Vanschoren
OpenML Benchmarking Suites. NeurIPS Datasets and Benchmarks (2021)
Aman Hussain, Nithin Holla, Pushkar Mishra, Helen Yannakoudakis, Ekaterina Shutova
Towards a robust experimental framework and benchmark for lifelong language learning. NeurIPS Datasets and Benchmarks (2021)
Haozhe Sun, Wei-Wei Tu, Isabelle Guyon
OmniPrint: A Configurable Printed Character Synthesizer. NeurIPS Datasets and Benchmarks (2021)
Thibaut Horel, Lorenzo Masoero, Raj Agrawal, Daria Roithmayr, Trevor Campbell
The CPD Data Set: Personnel, Use of Force, and Complaints in the Chicago Police Department. NeurIPS Datasets and Benchmarks (2021)
William Gilpin
Chaos as an interpretable benchmark for forecasting and data-driven modelling. NeurIPS Datasets and Benchmarks (2021)
Cédric Renggli, Luka Rimanic, Nora Hollenstein, Ce Zhang
Evaluating Bayes Error Estimators on Real-World Datasets with FeeBee. NeurIPS Datasets and Benchmarks (2021)
Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks 1, NeurIPS Datasets and Benchmarks 2021, December 2021, virtual. NeurIPS Datasets and Benchmarks (2021)
Scott Freitas, Yuxiao Dong, Joshua Neil, Duen Horng Chau
A Large-Scale Database for Graph Representation Learning. NeurIPS Datasets and Benchmarks (2021)
Jack Bandy, Nicholas Vincent
Addressing "Documentation Debt" in Machine Learning: A Retrospective Datasheet for BookCorpus. NeurIPS Datasets and Benchmarks (2021)
Laurynas Karazija, Iro Laina, Christian Rupprecht
ClevrTex: A Texture-Rich Benchmark for Unsupervised Multi-Object Segmentation. NeurIPS Datasets and Benchmarks (2021)
Saahil Jain, Ashwin Agrawal, Adriel Saporta, Steven Q. H. Truong, Du Nguyen Duong, Tan Bui, Pierre Chambon, Yuhao Zhang, Matthew P. Lungren, Andrew Y. Ng, Curtis P. Langlotz, Pranav Rajpurkar
RadGraph: Extracting Clinical Entities and Relations from Radiology Reports. NeurIPS Datasets and Benchmarks (2021)
Curtis G. Northcutt, Anish Athalye, Jonas Mueller
Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarks. NeurIPS Datasets and Benchmarks (2021)
Sudeep Dasari, Jianren Wang, Joyce Hong, Shikhar Bahl, Yixin Lin, Austin S. Wang, Abitha Thankaraj, Karanbir Chahal, Berk Çalli, Saurabh Gupta, David Held, Lerrel Pinto, Deepak Pathak, Vikash Kumar, Abhinav Gupta
RB2: Robotic Manipulation Benchmarking with a Twist. NeurIPS Datasets and Benchmarks (2021)
John Lambert, James Hays
Trust, but Verify: Cross-Modality Fusion for HD Map Change Detection. NeurIPS Datasets and Benchmarks (2021)
Daniel Galvez, Greg Diamos, Juan Torres, Keith Achorn, Juan Felipe Cerón, Anjali Gopi, David Kanter, Max Lam, Mark Mazumder, Vijay Janapa Reddi
The People's Speech: A Large-Scale Diverse English Speech Recognition Dataset for Commercial Usage. NeurIPS Datasets and Benchmarks (2021)
Tal Ridnik, Emanuel Ben Baruch, Asaf Noy, Lihi Zelnik
ImageNet-21K Pretraining for the Masses. NeurIPS Datasets and Benchmarks (2021)
Dan Hendrycks, Collin Burns, Anya Chen, Spencer Ball
CUAD: An Expert-Annotated NLP Dataset for Legal Contract Review. NeurIPS Datasets and Benchmarks (2021)
Moein Sorkhei, Yue Liu, Hossein Azizpour, Edward Azavedo, Karin Dembrower, Dimitra Ntoula, Athanasios Zouzos, Fredrik Strand, Kevin Smith
CSAW-M: An Ordinal Classification Dataset for Benchmarking Mammographic Masking of Cancer. NeurIPS Datasets and Benchmarks (2021)
Jianhua Han, Xiwen Liang, Hang Xu, Kai Chen, Lanqing Hong, Jiageng Mao, Chaoqiang Ye, Wei Zhang, Zhenguo Li, Xiaodan Liang, Chunjing Xu
SODA10M: A Large-Scale 2D Self/Semi-Supervised Object Detection Dataset for Autonomous Driving. NeurIPS Datasets and Benchmarks (2021)
William G. La Cava, Patryk Orzechowski, Bogdan Burlacu, Fabrício Olivetti de França, Marco Virgolin, Ying Jin, Michael Kommenda, Jason H. Moore
Contemporary Symbolic Regression Methods and their Relative Performance. NeurIPS Datasets and Benchmarks (2021)
Tongzhou Mu, Zhan Ling, Fanbo Xiang, Derek Yang, Xuanlin Li, Stone Tao, Zhiao Huang, Zhiwei Jia, Hao Su
ManiSkill: Generalizable Manipulation Skill Benchmark with Large-Scale Demonstrations. NeurIPS Datasets and Benchmarks (2021)
Aleksandar Botev, Andrew Jaegle, Peter Wirnsberger, Daniel Hennes, Irina Higgins
Which priors matter? Benchmarking models for learning latent dynamics. NeurIPS Datasets and Benchmarks (2021)
Alex Tamkin, Vincent Liu, Rongfei Lu, Daniel Fein, Colin Schultz, Noah D. Goodman
DABS: a Domain-Agnostic Benchmark for Self-Supervised Learning. NeurIPS Datasets and Benchmarks (2021)
Alicia Curth, David Svensson, James Weatherall, Mihaela van der Schaar
Really Doing Great at Estimating CATE? A Critical Look at ML Benchmarking Practices in Treatment Effect Estimation. NeurIPS Datasets and Benchmarks (2021)
Wenhu Chen, Xinyi Wang, William Yang Wang
A Dataset for Answering Time-Sensitive Questions. NeurIPS Datasets and Benchmarks (2021)
Jiyang Qi, Yan Gao, Yao Hu, Xinggang Wang, Xiaoyu Liu, Xiang Bai, Serge J. Belongie, Alan L. Yuille, Philip H. S. Torr, Song Bai
Occluded Video Instance Segmentation: Dataset and ICCV 2021 Challenge. NeurIPS Datasets and Benchmarks (2021)
Camille Garcin, Alexis Joly, Pierre Bonnet, Antoine Affouard, Jean-Christophe Lombardo, Mathias Chouet, Maximilien Servajean, Titouan Lorieul, Joseph Salmon
Pl@ntNet-300K: a plant image dataset with high label ambiguity and a long-tailed distribution. NeurIPS Datasets and Benchmarks (2021)
Ann Yuan, Daphne Ippolito, Vitaly Nikolaev, Chris Callison-Burch, Andy Coenen, Sebastian Gehrmann
SynthBio: A Case Study in Faster Curation of Text Datasets. NeurIPS Datasets and Benchmarks (2021)
Zihao Wang, Hang Yin, Yangqiu Song
Benchmarking the Combinatorial Generalizability of Complex Query Answering on Knowledge Graphs. NeurIPS Datasets and Benchmarks (2021)
Mingjie Li, Wenjia Cai, Rui Liu, Yuetian Weng, Xiaoyun Zhao, Cong Wang, Xin Chen, Zhong Liu, Caineng Pan, Mengke Li, Yingfeng Zheng, Yizhi Liu, Flora D. Salim, Karin Verspoor, Xiaodan Liang, Xiaojun Chang
FFA-IR: Towards an Explainable and Reliable Medical Report Generation Benchmark. NeurIPS Datasets and Benchmarks (2021)
Rami Aly, Zhijiang Guo, Michael Sejr Schlichtkrull, James Thorne, Andreas Vlachos, Christos Christodoulopoulos, Oana Cocarascu, Arpit Mittal
FEVEROUS: Fact Extraction and VERification Over Unstructured and Structured information. NeurIPS Datasets and Benchmarks (2021)
Tong Xia, Dimitris Spathis, Chloë Brown, Jagmohan Chauhan, Andreas Grammenos, Jing Han, Apinan Hasthanasombat, Erika Bondareva, Ting Dang, Andres Floto, Pietro Cicuta, Cecilia Mascolo
COVID-19 Sounds: A Large-Scale Audio Dataset for Digital Respiratory Screening. NeurIPS Datasets and Benchmarks (2021)
Salva Rühling Cachay, Venkatesh Ramesh, Jason N. S. Cole, Howard Barker, David Rolnick
ClimART: A Benchmark Dataset for Emulating Atmospheric Radiative Transfer in Weather and Climate Models. NeurIPS Datasets and Benchmarks (2021)
Lukas Kondmann, Aysim Toker, Marc Rußwurm, Andrés Camero, Devis Peressuti, Grega Milcinski, Pierre-Philippe Mathieu, Nicolas Longépé, Timothy Davis, Giovanni Marchisio, Laura Leal-Taixé, Xiaoxiang Zhu
DENETHOR: The DynamicEarthNET dataset for Harmonized, inter-Operable, analysis-Ready, daily crop monitoring from space. NeurIPS Datasets and Benchmarks (2021)
Zhaojiang Lin, Andrea Madotto, Genta Indra Winata, Peng Xu, Feijun Jiang, Yuxiang Hu, Chen Shi, Pascale Fung
BiToD: A Bilingual Multi-Domain Dataset For Task-Oriented Dialogue Modeling. NeurIPS Datasets and Benchmarks (2021)
Yuhang Li, Mingzhu Shen, Jian Ma, Yan Ren, Mingxin Zhao, Qi Zhang, Ruihao Gong, Fengwei Yu, Junjie Yan
MQBench: Towards Reproducible and Deployable Model Quantization Benchmark. NeurIPS Datasets and Benchmarks (2021)
Jaeju An, Jeongho Kim, Hanbeen Lee, Jinbeom Kim, Junhyung Kang, Minha Kim, Saebyeol Shin, Donghee Hong, Simon S. Woo
VFP290K: A Large-Scale Benchmark Dataset for Vision-based Fallen Person Detection. NeurIPS Datasets and Benchmarks (2021)
Dennis Assenmacher, Marco Niemann, Kilian Müller, Moritz Seiler, Dennis M. Riehle, Heike Trautmann
RP-Mod&RP-Crowd: Moderator- and Crowd-Annotated German News Comment Datasets. NeurIPS Datasets and Benchmarks (2021)
Karan Desai, Gaurav Kaul, Zubin Aysola, Justin Johnson
RedCaps: Web-curated image-text data created by the people, for the people. NeurIPS Datasets and Benchmarks (2021)
Alon Talmor, Ori Yoran, Ronan Le Bras, Chandra Bhagavatula, Yoav Goldberg, Yejin Choi, Jonathan Berant
CommonsenseQA 2.0: Exposing the Limits of AI through Gamification. NeurIPS Datasets and Benchmarks (2021)
Samriddhi Singla, Ayan Mukhopadhyay, Michael Wilbur, Tina Diao, Vinayak Gajjewar, Ahmed Eldawy, Mykel J. Kochenderfer, Ross D. Shachter, Abhishek Dubey
WildfireDB: An Open-Source Dataset Connecting Wildfire Occurrence with Relevant Determinants. NeurIPS Datasets and Benchmarks (2021)
Junjue Wang, Zhuo Zheng, Ailong Ma, Xiaoyan Lu, Yanfei Zhong
LoveDA: A Remote Sensing Land-Cover Dataset for Domain Adaptive Semantic Segmentation. NeurIPS Datasets and Benchmarks (2021)
Shuai Lu, Daya Guo, Shuo Ren, Junjie Huang, Alexey Svyatkovskiy, Ambrosio Blanco, Colin B. Clement, Dawn Drain, Daxin Jiang, Duyu Tang, Ge Li, Lidong Zhou, Linjun Shou, Long Zhou, Michele Tufano, Ming Gong, Ming Zhou, Nan Duan, Neel Sundaresan, Shao Kun Deng, Shengyu Fu, Shujie Liu
CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation. NeurIPS Datasets and Benchmarks (2021)
Maria Korosteleva, Sung-Hee Lee
Generating Datasets of 3D Garments with Sewing Patterns. NeurIPS Datasets and Benchmarks (2021)
Yanqiao Zhu, Yichen Xu, Qiang Liu, Shu Wu
An Empirical Study of Graph Contrastive Learning. NeurIPS Datasets and Benchmarks (2021)
Kenneth Peng, Arunesh Mathur, Arvind Narayanan
Mitigating dataset harms requires stewardship: Lessons from 1000 papers. NeurIPS Datasets and Benchmarks (2021)