Ian Soboroff

ORCID

Publication Activity (10 Years)

Years Active: 1996-2023
Publications (10 Years): 67

Top Venues

Publications

Keith Curtis, George Awad, Afzal Godil, Ian Soboroff
The ACM Multimedia 2023 Deep Video Understanding Grand Challenge. ACM Multimedia (2023)
Ian Soboroff
The BETTER Cross-Language Datasets. SIGIR (2023)
Tetsuya Sakai, Sijie Tao, Maria Maistro, Zhumin Chu, Yujing Li, Nuo Chen, Nicola Ferro, Junjie Wang, Ian Soboroff, Yiqun Liu
Corrected Evaluation Results of the NTCIR WWW-2, WWW-3, and WWW-4 English Subtasks. CoRR (2022)
Gareth J. F. Jones, Maria Eskevich, Ben Carterette, Joana Correia, Rosie Jones, Jussi Karlgren, Ian Soboroff
CHIIR Workshop on Audio Collection Human Interaction (AudioCHI 2022): http: //speechretrievalworkshop.github.io. CHIIR (2022)
Rezvaneh Rezapour, Sravana Reddy, Rosie Jones, Ian Soboroff
What Makes a Good Podcast Summary? SIGIR (2022)
Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, Jimmy Lin, Ellen M. Voorhees, Ian Soboroff
Overview of the TREC 2022 Deep Learning Track. TREC (2022)
Keith Curtis, George Awad, Shahzad Rajput, Ian Soboroff
Second International Workshop on Deep Video Understanding. ICMI (2022)
Cody Buntain, Richard McCreadie, Ian Soboroff
Incident Streams 2021 Off the Deep End: Deeper Annotations and Evaluations in Twitter. ISCRAM (2022)
Ellen M. Voorhees, Ian Soboroff, Jimmy Lin
Can Old TREC Collections Reliably Evaluate Modern Neural Retrieval Models? CoRR (2022)
Gareth J. F. Jones, Maria Eskevich, Ben Carterette, Joana Correia, Rosie Jones, Jussi Karlgren, Ian Soboroff
Report on the 1st Workshop on Audio Collection Human Interaction (AudioCHI 2022) at CHIIR 2022. SIGIR Forum 56 (1) (2022)
Keith Curtis, George Awad, Shahzad Rajput, Ian Soboroff
The ACM Multimedia 2022 Deep Video Understanding Grand Challenge. ACM Multimedia (2022)
Chao-Hong Liu, Alina Karakanta, Audrey Tong, Oleg Aulov, Ian Soboroff, Jonathan Washington, Xiaobing Zhao
Introduction to the second issue on machine translation for low-resource languages. Mach. Transl. 35 (1) (2021)
Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, Ellen M. Voorhees, Ian Soboroff
TREC Deep Learning Track: Reusable Test Collections in the Large Data Regime. SIGIR (2021)
Ian Soboroff
Overview of TREC 2021. TREC (2021)
Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, Ellen M. Voorhees, Ian Soboroff
TREC Deep Learning Track: Reusable Test Collections in the Large Data Regime. CoRR (2021)
Kirk Roberts, Tasmeer Alam, Steven Bedrick, Dina Demner-Fushman, Kyle Lo, Ian Soboroff, Ellen M. Voorhees, Lucy Lu Wang, William R. Hersh
Searching for scientific evidence in a pandemic: An overview of TREC-COVID. J. Biomed. Informatics 121 (2021)
Ben Carterette, Rosie Jones, Gareth J. F. Jones, Maria Eskevich, Sravana Reddy, Ann Clifton, Yongze Yu, Jussi Karlgren, Ian Soboroff
Podcast Metadata and Content: Episode Relevance andAttractiveness in Ad Hoc Search. CoRR (2021)
Kirk Roberts, Tasmeer Alam, Steven Bedrick, Dina Demner-Fushman, Kyle Lo, Ian Soboroff, Ellen M. Voorhees, Lucy Lu Wang, William R. Hersh
Searching for Scientific Evidence in a Pandemic: An Overview of TREC-COVID. CoRR (2021)
Ben Carterette, Rosie Jones, Gareth J. F. Jones, Maria Eskevich, Sravana Reddy, Ann Clifton, Yongze Yu, Jussi Karlgren, Ian Soboroff
Podcast Metadata and Content: Episode Relevance and Attractiveness in Ad Hoc Search. SIGIR (2021)
Cody Buntain, Richard McCreadie, Ian Soboroff
Incident Streams 2020: TRECIS in the Time of COVID-19. ISCRAM (2021)
Keith Curtis, George Awad, Shahzad Rajput, Ian Soboroff
International Workshop on Deep Video Understanding. ICMI (2020)
Timo Breuer, Nicola Ferro, Norbert Fuhr, Maria Maistro, Tetsuya Sakai, Philipp Schaer, Ian Soboroff
How to Measure the Reproducibility of System-oriented IR Experiments. SIGIR (2020)
Keith Curtis, George Awad, Shahzad Rajput, Ian Soboroff
HLVU: A New Challenge to Test Deep Understanding of Movies the Way Humans do. ICMR (2020)
Keith Curtis, George Awad, Shahzad Rajput, Ian Soboroff
HLVU : A New Challenge to Test Deep Understanding of Movies the Way Humans do. CoRR (2020)
Ellen M. Voorhees, Tasmeer Alam, Steven Bedrick, Dina Demner-Fushman, William R. Hersh, Kyle Lo, Kirk Roberts, Ian Soboroff, Lucy Lu Wang
TREC-COVID: constructing a pandemic information retrieval test collection. SIGIR Forum 54 (1) (2020)
Ian Soboroff, George Awad, Asad A. Butt, Keith Curtis
Evaluating Multimedia and Language Tasks. Frontiers Artif. Intell. 3 (2020)
Kirk Roberts, Tasmeer Alam, Steven Bedrick, Dina Demner-Fushman, Kyle Lo, Ian Soboroff, Ellen M. Voorhees, Lucy Lu Wang, William R. Hersh
TREC-COVID: rationale and structure of an information retrieval shared task for COVID-19. J. Am. Medical Informatics Assoc. 27 (9) (2020)
Ellen M. Voorhees, Tasmeer Alam, Steven Bedrick, Dina Demner-Fushman, William R. Hersh, Kyle Lo, Kirk Roberts, Ian Soboroff, Lucy Lu Wang
TREC-COVID: Constructing a Pandemic Information Retrieval Test Collection. CoRR (2020)
Cody Buntain, Richard McCreadie, Ian Soboroff
Incident Streams 2020: TRECIS in the Time of COVID-19. TREC (2020)
Chao-Hong Liu, Alina Karakanta, Audrey Tong, Oleg Aulov, Ian Soboroff, Jonathan Washington, Xiaobing Zhao
Introduction to the Special Issue on Machine Translation for Low-Resource Languages. Mach. Transl. 34 (4) (2020)
Richard McCreadie, Cody Buntain, Ian Soboroff
Incident Streams 2019: Actionable Insights and How to Find Them. ISCRAM (2020)
Ian Soboroff, Shudong Huang, Donna Harman
TREC 2020 News Track Overview. TREC (2020)
Timo Breuer, Nicola Ferro, Norbert Fuhr, Maria Maistro, Tetsuya Sakai, Philipp Schaer, Ian Soboroff
How to Measure the Reproducibility of System-oriented IR Experiments. CoRR (2020)
Louiqa Raschid, Douglas Burdick, Cesar de Pablo, Mark D. Flood, John Grant, Joe Langsam, Jay Pujara, Elena Tomas, Ian Soboroff
Financial Entity Identification and Information Integration (FEIII) 2019 Challenge: The Report of the Organizing Committee. DSMM@SIGMOD (2019)
Richard McCreadie, Cody Buntain, Ian Soboroff
TREC Incident Streams: Finding Actionable Information on Social Media. ISCRAM (2019)
Heng Ji, Avirup Sil, Hoa Trang Dang, Ian Soboroff, Joel Nothman
Overview of TAC-KBP 2019 Fine-grained Entity Extraction. TAC (2019)
Nicola Ferro, Norbert Fuhr, Maria Maistro, Tetsuya Sakai, Ian Soboroff
Overview of CENTRE@CLEF 2019: Sequel in the Systematic Reproducibility Realm. CLEF (2019)
Nicola Ferro, Ian Soboroff, Min Zhang
EVIA 2019 Preface. EVIA@NTCIR (2019)
Nicola Ferro, Norbert Fuhr, Maria Maistro, Tetsuya Sakai, Ian Soboroff
CENTRE@CLEF 2019. ECIR (2) (2019)
Richard McCreadie, Shahzad Rajput, Ian Soboroff, Craig Macdonald, Iadh Ounis
On enhancing the robustness of timeline summarization test collections. Inf. Process. Manag. 56 (5) (2019)
Nicola Ferro, Norbert Fuhr, Maria Maistro, Tetsuya Sakai, Ian Soboroff
CENTRE@CLEF2019: Overview of the Replicability and Reproducibility Tasks. CLEF (Working Notes) (2019)
Nicola Ferro, Ian Soboroff
Report on EVIA 2017. SIGIR Forum 52 (1) (2018)
Ian Soboroff, Shudong Huang, Donna Harman
TREC 2018 News Track Overview. TREC (2018)
Nicola Ferro, Maria Maistro, Tetsuya Sakai, Ian Soboroff
CENTRE@CLEF2018: Overview of the Replicability Task. CLEF (Working Notes) (2018)
Nicola Ferro, Maria Maistro, Tetsuya Sakai, Ian Soboroff
Overview of CENTRE@CLEF 2018: A First Tale in the Systematic Reproducibility Realm. CLEF (2018)
Audrey Tong, Lukas L. Diduch, Jonathan G. Fiscus, Yasaman Haghpanah, Shudong Huang, David Joy, Kay Peterson, Ian Soboroff
Overview of the NIST 2016 LoReHLT evaluation. Mach. Transl. 32 (1-2) (2018)
Ian Soboroff
Meta-Analysis for Retrieval Experiments Involving Multiple Test Collections. CIKM (2018)
Shudong Huang, Ian Soboroff, Donna Harman
TREC 2018 News Track. NewsIR@ECIR (2018)
Ian Soboroff, Audrey Tong
Preface. Mach. Transl. 32 (1-2) (2018)
Louiqa Raschid, Douglas Burdick, John Grant, Joe Langsam, Jay Pujara, Elizabeth Roman, Ian Soboroff, Mohammed J. Zaki, Elena Zotkina
Financial Entity Identification and Information Integration (FEIII) 2018 Challenge: The Report of the Organizing Committee. DSMM@SIGMOD (2018)
Ian Soboroff, Nicola Ferro, Norbert Fuhr
Report on GLARE 2018: 1st Workshop on Generalization in Information Retrieval. SIGIR Forum 52 (2) (2018)
Nicola Ferro, Norbert Fuhr, Gregory Grefenstette, Joseph A. Konstan, Pablo Castells, Elizabeth M. Daly, Thierry Declerck, Michael D. Ekstrand, Werner Geyer, Julio Gonzalo, Tsvi Kuflik, Krister Lindén, Bernardo Magnini, Jian-Yun Nie, Raffaele Perego, Bracha Shapira, Ian Soboroff, Nava Tintarev, Karin Verspoor, Martijn C. Willemsen, Justin Zobel
The Dagstuhl Perspectives Workshop on Performance Modeling and Prediction. SIGIR Forum 52 (1) (2018)
Nicola Ferro, Norbert Fuhr, Gregory Grefenstette, Joseph A. Konstan, Pablo Castells, Elizabeth M. Daly, Thierry Declerck, Michael D. Ekstrand, Werner Geyer, Julio Gonzalo, Tsvi Kuflik, Krister Lindén, Bernardo Magnini, Jian-Yun Nie, Raffaele Perego, Bracha Shapira, Ian Soboroff, Nava Tintarev, Karin Verspoor, Martijn C. Willemsen, Justin Zobel
From Evaluating to Forecasting Performance: How to Turn Information Retrieval, Natural Language Processing and Recommender Systems into Predictive Sciences (Dagstuhl Perspectives Workshop 17442). Dagstuhl Manifestos 7 (1) (2018)
Ian Soboroff
New Tracks for TREC 2018. NTCIR (2017)
Ian Soboroff
Building Test Collections: An Interactive Guide for Students and Others Without Their Own Evaluation Conference Series. SIGIR (2017)
Ellen M. Voorhees, Daniel Samarov, Ian Soboroff
Using Replicates in Information Retrieval Evaluation. ACM Trans. Inf. Syst. 36 (2) (2017)
Louiqa Raschid, Douglas Burdick, Mark D. Flood, John Grant, Joe Langsam, Ian Soboroff
Financial Entity Identification and Information Integration (FEIII) 2017 Challenge: The Report of the Organizing Committee. DSMM@SIGMOD (2017)
Grace Hui Yang, Zhiwen Tang, Ian Soboroff
TREC 2017 Dynamic Domain Track Overview. TREC (2017)
Ian Soboroff, Kira Griffitt, Stephanie M. Strassel
The BOLT IR Test Collections of Multilingual Passage Retrieval from Discussion Forums. SIGIR (2016)
Grace Hui Yang, Ian Soboroff
TREC 2016 Dynamic Domain Track Overview. TREC (2016)
Mark D. Flood, John Grant, Haiping Luo, Louiqa Raschid, Ian Soboroff, Kyungjin Yoo
Financial Entity Identification and Information Integration (FEIII) Challenge: The Report of the Organizing Committee. DSMM@SIGMOD (2016)
Ellen M. Voorhees, Shahzad Rajput, Ian Soboroff
Promoting Repeatability Through Open Runs. EVIA@NTCIR (2016)
Grace Hui Yang, Ian Soboroff, Li Xiong, Charles L. A. Clarke, Simson L. Garfinkel
Privacy-Preserving IR 2016: Differential Privacy, Search, and Social Media. SIGIR (2016)
Ian Soboroff
News from TREC. NTCIR (2016)
Grace Hui Yang, Ian Soboroff
Privacy Preserving IR 2015: A SIGIR 2015 Workshop. SIGIR Forum 49 (2) (2015)
Hui Yang, John R. Frank, Ian Soboroff
TREC 2015 Dynamic Domain Track Overview. TREC (2015)
Hui Yang, Ian Soboroff
Privacy-Preserving IR 2015: When Information Retrieval Meets Privacy and Security. SIGIR (2015)
Ian Soboroff
Computing Confidence Intervals for Common IR Measures. EVIA@NTCIR (2014)
Ellen M. Voorhees, Paul Over, Ian Soboroff
Building Better Search Engines by Measuring Search Quality. IT Prof. 16 (2) (2014)
John R. Frank, Max Kleiman-Weiner, Daniel A. Roberts, Ellen M. Voorhees, Ian Soboroff
Evaluating Stream Filtering for Entity Profile Updates in TREC 2012, 2013, and 2014. TREC (2014)
Ian Soboroff
Building test collections: an interactive tutorial for students and others without their own evaluation conference series. SIGIR (2013)
Tetsuya Sakai, Noriko Kando, Craig Macdonald, Ian Soboroff
Introduction to the special issue on search intents and diversification. Inf. Retr. 16 (4) (2013)
John R. Frank, Steven J. Bauer, Max Kleiman-Weiner, Daniel A. Roberts, Nilesh Tripuraneni, Ce Zhang, Christopher Ré, Ellen M. Voorhees, Ian Soboroff
Evaluating Stream Filtering for Entity Profile Updates for TREC 2013. TREC (2013)
Ian Soboroff, Iadh Ounis, Craig Macdonald, Jimmy Lin
Overview of the TREC-2012 Microblog Track. TREC (2012)
Rodrygo L. T. Santos, Craig Macdonald, Richard McCreadie, Iadh Ounis, Ian Soboroff
Information Retrieval on the Blogosphere. Found. Trends Inf. Retr. 6 (1) (2012)
John R. Frank, Max Kleiman-Weiner, Daniel A. Roberts, Feng Niu, Ce Zhang, Christopher Ré, Ian Soboroff
Building an Entity-Centric Stream Filtering Test Collection for TREC 2012. TREC (2012)
Richard McCreadie, Ian Soboroff, Jimmy Lin, Craig Macdonald, Iadh Ounis, Dean McCullough
On building a reusable Twitter corpus. SIGIR (2012)
Iadh Ounis, Craig Macdonald, Jimmy Lin, Ian Soboroff
Overview of the TREC 2011 Microblog Track. TREC (2011)
Charles L. A. Clarke, Nick Craswell, Ian Soboroff, Azin Ashkan
A comparative analysis of cascade measures for novelty and diversity. WSDM (2011)
Charles L. A. Clarke, Nick Craswell, Ian Soboroff, Ellen M. Voorhees
Overview of the TREC 2011 Web Track. TREC (2011)
Mark Sanderson, William Webber, Ian Soboroff, Gareth J. F. Jones, Andrew Trotman, Shlomo Geva, Nicola Ferro, Hideo Joho
Reports from Other Evaluation Campaigns and Panel Discussion: TREC is 20 Years Old, Where Now for Evaluation Campaigns? EVIA@NTCIR (2011)
Craig Macdonald, Rodrygo L. T. Santos, Iadh Ounis, Ian Soboroff
Blog track research at TREC. SIGIR Forum 44 (1) (2010)
Ian Soboroff
Test Collection Diagnosis and Treatment. EVIA@NTCIR (2010)
Ben Carterette, Ian Soboroff
The effect of assessor error on IR system evaluation. SIGIR (2010)
Iadh Ounis, Ian Soboroff
Overview of the TREC 2010 Blog Track. TREC (2010)
Charles L. A. Clarke, Nick Craswell, Ian Soboroff, Gordon V. Cormack
Overview of the TREC 2010 Web Track. TREC (2010)
Craig Macdonald, Iadh Ounis, Ian Soboroff
Is spam an issue for opinionated blog post search? SIGIR (2009)
Ian Soboroff
A guide to the RIA workshop data archive. Inf. Retr. 12 (6) (2009)
Eugene Agichtein, Marti A. Hearst, Ian Soboroff
The search and social media workshop at SIGIR 2009. SIGIR Forum 43 (2) (2009)
Craig Macdonald, Iadh Ounis, Ian Soboroff
Overview of the TREC 2009 Blog Track. TREC (2009)
Charles L. A. Clarke, Nick Craswell, Ian Soboroff
Overview of the TREC 2009 Web Track. TREC (2009)
Iadh Ounis, Craig MacDonald, Ian Soboroff
Overview of the TREC 2008 Blog Track. TREC (2008)
Ian Soboroff
TREC, TAC, takeoffs, tacks, tasks, and titillations for 2009. NTCIR (2008)
Craig Macdonald, Ben He, Iadh Ounis, Ian Soboroff
Limits of opinion-finding baseline systems. SIGIR (2008)
Peter Bailey, Nick Craswell, Ian Soboroff, Paul Thomas, Arjen P. de Vries, Emine Yilmaz
Relevance assessment: are judges exchangeable and does it matter. SIGIR (2008)
Iadh Ounis, Craig Macdonald, Ian Soboroff
On the TREC Blog Track. ICWSM (2008)
Krisztian Balog, Ian Soboroff, Paul Thomas, Peter Bailey, Nick Craswell, Arjen P. de Vries
Overview of the TREC 2008 Enterprise Track. TREC (2008)
Chris Buckley, Darrin Dimmick, Ian Soboroff, Ellen M. Voorhees
Bias and the limits of pooling for large collections. Inf. Retr. 10 (6) (2007)
Ian Soboroff
What's New at TREC: Blog and Legal Discovery Search at TREC-2006. NTCIR (2007)
Ian Soboroff
A comparison of pooled and sampled relevance judgments. SIGIR (2007)

Ian Soboroff.css-eqlu8i{font-size:small;padding-left:4px;}

Publication Activity (10 Years)

Top Topics

Top Venues

Ian Soboroff