On the Evaluation of Dynamic Critiquing: A Large-Scale User Study

Abstract

Critiquing is an important form of feedback in conver-sational recommender systems. However, in these sys-tems the user is usually limited to critiquing a single product feature at a time. Recently dynamic critiquing has been proposed to address this shortcoming, by auto-matically generating compound critiques over multiple features that may be presented to the user at recommen-dation time. To date a number of different versions of dynamic critiquing have been evaluated in isolation, and with reference to artificial users. In this paper we bring together the main flavors of dynamic critiquing and per-form a large-scale comparative evaluation as part of an extensive real-user trial. This evaluation reveals some interesting facts about the way real users interact with critique-based recommenders.