预测对话推荐系统何时会失败,提前识别用户需求无法满足的情况。
Failure Prediction in Conversational Recommendation Systems
- 基于多轮语义信息构建自编码器预测器,判断对话是否失败。
- 在鞋类和时尚连衣裙数据集上,系统失败预测准确率达78.3%。
- 适用于希望提升对话推荐鲁棒性的研究者与工程师。
在对话式图像推荐任务中,用户可通过自然语言反馈优化后续推荐。然而,若目标物品不在商品库中或系统无法找到,可能导致用户频繁交互并产生挫败感。为缓解此问题,本文提出监督式对话性能预测任务,受查询性能预测(QPP)启发,利用多轮对话中检索到图像项的嵌入表示中的语义信息,检测对话失败。我们提出基于自编码器的预测模型,学习训练轮次中前Top项的压缩表示,并用分类标签预测评估轮次的表现。实验在Shoes和FashionIQ Dresses数据集上进行,区分系统失败(系统找不到目标)与目录失败(目标不存在于目录)。结果显示,系统失败预测表现良好(准确率78.3%),而目录失败预测性能显著下降,表明该任务在真实场景中仍有挑战。
原文摘要 · Abstract (English)
In a Conversational Image Recommendation task, users can provide natural language feedback on a recommended image item, which leads to an improved recommendation in the next turn. While typical instantiations of this task assume that the user's target item will (eventually) be returned, this might often not be true, for example, the item the user seeks is not within the item catalogue. Failing to return a user's desired item can lead to user frustration, as the user needs to interact with the system for an increased number of turns. To mitigate this issue, in this paper, we introduce the task of Supervised Conversational Performance Prediction, inspired by Query Performance Prediction (QPP) for predicting effectiveness in response to a search engine query. In this regard, we propose predictors for conversational performance that detect conversation failures using multi-turn semantic information contained in the embedded representations of retrieved image items. Specifically, our AutoEncoder-based predictor learns a compressed representation of top-retrieved items of the train turns and uses the classification labels to predict the evaluation turn. Our evaluation scenario addressed two recommendation scenarios, by differentiating between system failure, where the system is unable to find the target, and catalogue failure, where the target does not exist in the item catalogue. In our experiments using the Shoes and FashionIQ Dresses datasets, we measure the accuracy of predictors for both system and catalogue failures. Our results demonstrate the promise of our proposed predictors for predicting system failures (existing evaluation scenario), while we detect a considerable decrease in predictive performance in the case of catalogue failure prediction (when inducing a missing item scenario) compared to system failures.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。