用认知模型解析用户理解AI解释的思维过程，提升可解释AI实用性。
CoAX: Cognitive-Oriented Attribution eXplanation User Model of Human Understanding of AI Explanations
- 基于用户研究构建认知模型，模拟不同解释方式下的推理策略。
- 模型拟合度优于传统机器学习方法，揭示有效与无效的推理路径。
- 可低成本验证人类理解机制，助力设计更易用的AI解释系统。
可解释人工智能（XAI）旨在提升用户在使用AI模型时的理解与决策能力。然而，尽管XAI技术不断进步，近期用户评估仍表明该目标难以实现。理解人类认知有助于揭示用户为何难以有效利用AI解释。本文聚焦结构化数据上的推理任务，研究了在预测AI决策（即前向模拟）中，无解释、特征重要性与特征归因等不同XAI方法所引发的多种推理策略。首先通过形成性用户研究提取推理策略，其次通过总结性用户研究收集实际决策数据。利用认知建模，实现各推理策略的心理过程并评估其与人类决策的一致性。结果表明，本模型比基线机器学习代理更贴合人类决策，揭示了哪些推理策略更有效。进一步展示如何利用拟合模型提出假设并探究成本高昂的人类实验问题。本研究为调试人类对XAI的理解提供支持，推动更具可用性与可解释性的未来解释系统发展。
原文摘要 · Abstract (English)
Explainable AI (XAI) aims to improve user understanding and decisions when using AI models. However, despite innovations in XAI, recent user evaluations reveal that this goal remains elusive. Understanding human cognition can help explain why users struggle to effectively use AI explanations. Focusing on reasoning on structured (tabular) data, we examined various reasoning strategies for different XAI methods (none, feature importance, feature attribution) in the decision task of anticipating AI decisions (i.e., forward simulation). We i) elicited reasoning strategies from a formative user study, and ii) collected decisions from a summative user study. Using cognitive modeling, we implemented the processes underlying each reasoning strategy and evaluated their alignment with human decision-making. We found that our models better fit human decisions than baseline machine learning proxies, providing insights into which reasoning strategies are (in)effective. We then demonstrate how the fitted model can be used to form hypotheses and investigate research questions that are costly to study with real human participants. This work contributes to debugging human understanding of XAI, informing the future development of more usable and interpretable AI explanations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。