用自适应分析提升人对AI决策的信任与使用效果
From Text to Trust: Empowering AI-assisted Decision Making with Adaptive LLM-powered Analysis
- 根据人类决策反应动态选择展示的AI分析内容
- 实验证明自适应策略显著提升人对AI的合理依赖度
- 适合需要提升人机协作效率的AI应用设计者
AI辅助决策日益普及，但人们常因缺乏AI解释而无法正确使用。大型语言模型（LLMs）具备出色的对话与分析能力，可在无解释时提供自然语言形式的决策分析，如各特征如何影响AI推荐。本研究通过随机实验发现，无论顺序或并行展示特征分析，均未显著提升决策表现。为此，我们提出一种算法框架，用于评估LLM分析对人类决策的影响，并动态决定展示内容。真人实验表明，该方法有效提升了决策者对AI的合理依赖程度。
原文摘要 · Abstract (English)
AI-assisted decision making becomes increasingly prevalent, yet individuals often fail to utilize AI-based decision aids appropriately especially when the AI explanations are absent, potentially as they do not %understand reflect on AI's decision recommendations critically. Large language models (LLMs), with their exceptional conversational and analytical capabilities, present great opportunities to enhance AI-assisted decision making in the absence of AI explanations by providing natural-language-based analysis of AI's decision recommendation, e.g., how each feature of a decision making task might contribute to the AI recommendation. In this paper, via a randomized experiment, we first show that presenting LLM-powered analysis of each task feature, either sequentially or concurrently, does not significantly improve people's AI-assisted decision performance. To enable decision makers to better leverage LLM-powered analysis, we then propose an algorithmic framework to characterize the effects of LLM-powered analysis on human decisions and dynamically decide which analysis to present. Our evaluation with human subjects shows that this approach effectively improves decision makers' appropriate reliance on AI in AI-assisted decision making.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。