Relevance Judgment Convergence Degree - A Measure of Inconsistency among Assessors for Information Retrieval.
Dengya ZhuShastri L. NimmagaddaKok Wai WongTorsten ReinersPublished in: ISD (2022)
Keyphrases
- information retrieval
- test collection
- document relevance
- relevance judgments
- relevant documents
- learning to rank
- relevant and non relevant documents
- information retrieval systems
- relevance feedback
- relevance assessments
- vector space model
- relevance judgements
- search engine
- relevance ranking
- retrieval systems
- document retrieval
- similarity measure
- document collections
- language model
- retrieval model
- retrieval effectiveness
- probability measure
- average precision
- convergence rate
- human subjects
- query expansion
- distance measure
- web search
- ranked list
- evaluation measures
- information access
- correlation coefficient
- text retrieval
- question answering
- text mining
- information retrieval evaluation
- ir evaluation
- discounted cumulative gain
- information extraction
- maximal marginal relevance