Obtaining High-Quality Relevance Judgments Using Crowdsourcing.
Jeroen B. P. VuurensArjen P. de VriesPublished in: IEEE Internet Comput. (2012)
Keyphrases
- relevance judgments
- high quality
- user feedback
- test collection
- retrieval systems
- web search engines
- learning to rank
- average precision
- retrieval effectiveness
- relevance feedback
- relevant documents
- ranking functions
- evaluation measures
- implicit feedback
- information retrieval
- evaluation metrics
- clickthrough data
- retrieval functions
- information retrieval evaluation
- query performance prediction
- user interaction
- queries and relevance judgments
- search engine
- user preferences
- relevance assessments
- document collections
- recommender systems
- user profiles
- information retrieval systems
- human relevance judgments