研究发现用户仍易过度依赖聊天机器人,即使能查证。
The Decision to Verify: How Warmth and User Characteristics Shape Reliance on Conversational Agents for Information Search

- 对比温暖与中性对话风格的聊天机器人,考察用户是否验证答案。
- 多数人不查证,仅少数人无论上下文都查证,多数人默认信任。
- 聊天机器人越温暖,用户越易接受错误答案,适合设计可信系统者参考。
对话式人工智能为信息获取提供了高效便捷的入口。然而,当用户盲目信任并接受其回答而不核实真实情况时,容易产生过度依赖。当前的信息搜索越来越多地采用融合对话式AI与网页搜索的混合交互模式,使事实核查更简便。本文探究该模式能否有效减少依赖,并分析驱动用户核实行为的关键因素(如数字素养、对话温度)。通过混合被试的问答实验,参与者与温暖或中性的聊天机器人互动。结果显示,尽管可同时使用对话式和网页搜索,过度依赖依然存在。是否核实主要取决于用户既有认知(如对聊天机器人的先期信任),而非回答本身特性:部分用户无论情境均会核实,另一些则默认信任。温暖对话风格虽不直接影响核实意愿,但会增加用户在机器人出错时的认同度。使用其他AI源可提升准确性,传统网页搜索则无显著效果。本研究拓展了过度依赖研究:(a) 表明即便有查证手段,依赖仍持续;(b) 揭示核实行为具有高度个体差异性;(c) 发现对话温度对过度依赖有间接影响,为构建可信对话式搜索系统提供启示。
原文摘要 · Abstract (English)
Conversational artificial intelligence (AI) provides an efficient and convenient gateway to information access. However, it can cause overreliance when users blindly trust AI and accept its answers without fact-checking. Information search increasingly follows a hybrid interaction paradigm that combines conversational AI with web search, making fact-checking easier. In this paper, we examine whether this interaction paradigm is effective in curbing reliance. We further investigate the underlying factors (e.g., digital literacy and conversation warmth) that drive users to verify AI answers. We conduct a mixed-subjects question-answering experiment where participants interact with either a warm or a neutral chatbot. Our findings reveal that reliance persists despite users having access to both conversational and web search. The decision to verify is driven primarily by existing user perceptions (e.g., prior trust in chatbots) rather than answer properties, with some users fact-checking regardless of the context and others trusting chatbots by default. Warm conversational style has an indirect yet critical influence on reliance by increasing agreement with the chatbot when it is incorrect. Consulting additional AI sources predicts higher accuracy, while traditional web search does not. Our study extends overreliance research by: (a) demonstrating its persistence despite access to fact-checking, (b) identifying verification behavior as user-dependent, and (c) revealing conversational warmth's indirect effect on overreliance with implications for designing trustworthy conversational search systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。