用AI叙事解释辅助决策,发现效果不显著还可能拖慢判断。
Human Decision-Making with Persuasive and Narrative LLM Explanations

- 用不同说服力的AI叙事解释人类分类决策
- 叙事解释未提升准确率,反而延长反应时间
- 适合关注人机交互中解释设计的研究者
大型语言模型(LLMs)在分类任务中不仅能提供准确预测,还能生成有说服力的叙事解释。以往研究表明,人们普遍认为这些解释易于理解、可信且能改变观点;但对客观决策表现的影响仍不明确。本文通过大规模人类行为实验,评估了不同说服力的LLM叙事解释对决策表现的影响。结果表明,叙事解释的说服力强弱对决策准确性无显著影响,与基于特征重要性的可解释AI结果一致。同时发现,无论预测正确与否,叙事解释都增加了人们对AI的依赖。探索性分析显示,更具说服力的叙事可能导致反应时间变长,并削弱区分正确与错误预测的能力。总体表明,加入叙事解释可能带来权衡,需进一步研究其适用场景与机制。
原文摘要 · Abstract (English)
Large language models (LLMs) have the potential to aid and improve human decision-making in classification tasks, not only by providing fairly accurate predictions, but also in their ability to generate cogent narrative explanations of those predictions. Prior work has demonstrated that people generally find AI narrative explanations to be understandable, trustworthy, and convincing for changing beliefs and opinions; however, less is known about the impact of narrative explanations on objective human decision-making performance. Here we conduct a large-scale human behavioral experiment to evaluate decision-making performance with LLM-generated narrative explanations of varying persuasiveness. We found the degree of persuasiveness, or lack thereof, for LLM-based explanations did not meaningfully impact decision accuracy over a simple AI prediction alone, in agreement with typical results with explainable AI based on feature importance. We found evidence that narratives increased reliance on AI, but both when the AI prediction was correct and incorrect. Exploratory analyses also indicated that the more persuasive narratives may have had a detrimental effect on decision response times and the ability to discriminate between a correct and incorrect AI prediction. Overall, this work indicates that including narrative explanations with AI predictions may involve tradeoffs for decision-making performance, and more work is needed to determine how and when narrative explanations impact human decision-making.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。