Incentivizing evaluation with peer prediction and limited access to ground truth.
Xi Alice GaoJames R. WrightKevin Leyton-BrownPublished in: Artif. Intell. (2019)
Keyphrases
- ground truth
- gold standard
- quantitative evaluation
- access control
- objective evaluation
- high quality
- prediction model
- evaluation process
- evaluation methods
- prediction error
- prediction accuracy
- prediction algorithm
- multiscale
- decision trees
- peer to peer systems
- data sets
- manually labeled
- bit rate
- information retrieval systems
- information systems
- machine learning