用户觉得像自己，就更喜欢聊天机器人。
How Stylistic Similarity Shapes Preferences in Dialogue Dataset with User and Third Party Evaluations
- 用用户自评和第三方评估区分主观与客观风格相似性。
- 用户主观相似性与偏好正相关，但与第三方评估不一致。
- 适合研究人机对话中风格匹配对体验的影响。
近期对话生成技术拓展了人机交互的范围，不仅支持上下文相关的回复，还能分析人类情感与敏感度。尽管已有研究指出用户与系统间风格相似性可能提升用户体验，但主观与客观相似性的区别常被忽视。为此，本文构建了一个新数据集，包含用户偏好、用户自我感知的主观风格相似性以及第三方评估者标注的客观风格相似性，涵盖开放域对话场景。分析显示，主观风格相似性与用户偏好存在显著正相关。此外，研究发现用户主观感知的相似性与其第三方客观评估结果存在差异，凸显区分两类评价的重要性，有助于理解风格相似性与用户偏好关系中不同评价维度所捕捉的特性。论文提供的数据集已公开可获取。
原文摘要 · Abstract (English)
Recent advancements in dialogue generation have broadened the scope of human-bot interactions, enabling not only contextually appropriate responses but also the analysis of human affect and sensitivity. While prior work has suggested that stylistic similarity between user and system may enhance user impressions, the distinction between subjective and objective similarity is often overlooked. To investigate this issue, we introduce a novel dataset that includes users' preferences, subjective stylistic similarity based on users' own perceptions, and objective stylistic similarity annotated by third party evaluators in open-domain dialogue settings. Analysis using the constructed dataset reveals a strong positive correlation between subjective stylistic similarity and user preference. Furthermore, our analysis suggests an important finding: users' subjective stylistic similarity differs from third party objective similarity. This underscores the importance of distinguishing between subjective and objective evaluations and understanding the distinct aspects each captures when analyzing the relationship between stylistic similarity and user preferences. The dataset presented in this paper is available online.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。