Mix and Match: Collaborative Expert-Crowd Judging for Building Test Collections Accurately and Affordably.
Mücahid KutluTyler McDonnellAashish SheshadriTamer ElsayedMatthew LeasePublished in: DESIRES (2018)
Keyphrases
- test collection
- information retrieval
- relevant documents
- retrieval model
- retrieval effectiveness
- language model
- information retrieval evaluation
- relevance judgments
- document collections
- average precision
- retrieval systems
- evaluation methodology
- relevance assessments
- ir evaluation
- evaluation of information retrieval systems
- ad hoc retrieval
- patent retrieval
- relevance judgements
- clef evaluation campaign
- xml retrieval
- newspaper articles
- chinese web
- document set
- evaluation campaigns
- test set
- pairwise
- learning algorithm