Examining the Limits of Crowdsourcing for Relevance Assessment.
Paul D. CloughMark SandersonJiayu TangTim GollinsAmy WarnerPublished in: IEEE Internet Comput. (2013)
Keyphrases
- relevance assessments
- interactive information retrieval
- user feedback
- test collection
- information seeking
- information retrieval systems
- retrieval systems
- evaluation metrics
- amazon mechanical turk
- information retrieval
- relevant documents
- ir evaluation
- machine learning
- retrieval effectiveness
- user interaction
- relevance judgments
- user studies
- user preferences
- average precision
- pairwise