Effects of objective and subjective competence on the reliability of crowdsourced relevance judgments.
Parnia SamimiSri Devi RavanaWilliam WebberYun Sing KohPublished in: Inf. Res. (2017)
Keyphrases
- relevance judgments
- test collection
- human judgments
- web search engines
- average precision
- retrieval systems
- learning to rank
- retrieval effectiveness
- relevance feedback
- user feedback
- ranking functions
- relevance assessments
- relevant documents
- evaluation measures
- evaluation metrics
- information retrieval
- clickthrough data
- information retrieval evaluation
- query expansion
- query performance prediction
- user queries
- retrieval functions
- information retrieval systems
- co occurrence
- pairwise
- query terms
- query logs
- retrieval model
- language model
- web search
- metadata