Weighted Krippendorff's alpha is a more reliable metrics for multi-coders ordinal annotations: experimental studies on emotion, opinion and coreference annotation.
Jean-Yves AntoineJeanne VillaneauAnaïs LefeuvrePublished in: EACL (2014)
Keyphrases
- experimental study
- semantic annotation
- annotation tool
- image annotation
- metadata
- manual annotation
- inter annotator agreement
- coding scheme
- automatic annotation
- experimental evaluation
- bit rate
- named entities
- active learning
- information extraction
- semantic web
- subband
- image coding
- automatic image annotation
- rank correlation
- video material
- mechanical turk
- reference resolution
- image coder
- semi automatic
- real life
- image retrieval