Rethinking the Evaluation of Dialogue Systems: Effects of User Feedback on Crowdworkers and LLMs.
Clemencia SiroMohammad AliannejadiMaarten de RijkePublished in: CoRR (2024)
Keyphrases
- user feedback
- dialogue system
- relevance judgments
- user interaction
- relevance feedback
- interactive retrieval
- natural language
- dialogue management
- implicit feedback
- user profiles
- user preferences
- learning to rank
- mixed initiative
- spoken dialogue systems
- recommender systems
- data sets
- amazon mechanical turk
- knowledge acquisition
- digital libraries
- tutorial dialogue