解释、来源和不一致性影响用户对大模型的信任程度。
Fostering Appropriate Reliance on Large Language Models: The Role of Explanations, Sources, and Inconsistencies
- 用解释增强用户对答案的信赖，无论答案对错。
- 提供来源或解释矛盾可降低对错误答案的依赖。
- 适合设计大模型交互界面时参考，提升用户判断力。
大型语言模型（LLMs）可能生成看似流畅且有说服力的错误回答，导致用户过度依赖。通过一项思考过程研究（参与者使用集成LLM的应用解答客观问题），我们识别出影响用户依赖度的三个响应特征：解释（支持性细节）、解释中的不一致性以及来源。在一项大规模、预先注册的受控实验中（N=308），我们分别考察这些特征对用户依赖度、准确率等的影响。结果显示，解释的存在会增加用户对正确和错误答案的依赖；但当提供来源或解释存在不一致时，用户对错误答案的依赖显著下降。这些发现对促进用户对LLM的合理依赖具有重要启示。
原文摘要 · Abstract (English)
Large language models (LLMs) can produce erroneous responses that sound fluent and convincing, raising the risk that users will rely on these responses as if they were correct. Mitigating such overreliance is a key challenge. Through a think-aloud study in which participants use an LLM-infused application to answer objective questions, we identify several features of LLM responses that shape users' reliance: explanations (supporting details for answers), inconsistencies in explanations, and sources. Through a large-scale, pre-registered, controlled experiment (N=308), we isolate and study the effects of these features on users' reliance, accuracy, and other measures. We find that the presence of explanations increases reliance on both correct and incorrect responses. However, we observe less reliance on incorrect responses when sources are provided or when explanations exhibit inconsistencies. We discuss the implications of these findings for fostering appropriate reliance on LLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。