Mix and Match: Collaborative Expert-Crowd Judging for Building Test Collections Accurately and Affordably.
Mücahid KutluTyler McDonnellAashish SheshadriTamer ElsayedMatthew LeasePublished in: CoRR (2018)
Keyphrases
- test collection
- relevant documents
- retrieval effectiveness
- average precision
- retrieval model
- evaluation methodology
- information retrieval
- language model
- document collections
- information retrieval evaluation
- retrieval systems
- relevance judgments
- ad hoc retrieval
- ir evaluation
- relevance assessments
- newspaper articles
- learning to rank
- evaluation of information retrieval systems
- chinese web
- relevance judgements
- clef evaluation campaign
- statistically significant
- evaluation campaigns
- databases
- xml retrieval
- e learning
- probabilistic model
- query terms