Agreement between Crowdsourced Workers and Expert Assessors in Making Relevance Judgment for System Based IR Evaluation.
Parnia SamimiSri Devi RavanaPublished in: SCDM (2014)
Keyphrases
- relevance assessments
- ir evaluation
- test collection
- relevance judgments
- information seeking
- information retrieval systems
- retrieval effectiveness
- information retrieval
- user feedback
- average precision
- evaluation metrics
- retrieval systems
- xml retrieval
- relevant documents
- retrieval model
- search tasks
- language model
- co occurrence
- information extraction