arXiv:2411.02419cs.CYcs.AI2024-11被引 5

构建了多领域用户对AI解释可理解性的研究数据集。

Dataset resulting from the user study on comprehensibility of explainable AI algorithms

  • 招募三个群体开展蘑菇分类模型解释的访谈研究。
  • 包含39份访谈转录本及解释可视化、主题分析等补充数据。
  • 适合研究可解释AI的跨学科评估与改进方案设计者。

本文介绍了一个用户研究生成的数据集,旨在评估可解释人工智能(XAI)算法的可理解性。研究从149名候选人中招募参与者,分为三组:真菌学专家(DE)、具备数据科学与可视化背景的学生(IT),以及社会科学研究类学生(SSH)。数据集核心部分包含39份访谈转录文本,参与者在任务中需解读一个用于区分可食用与不可食用蘑菇的机器学习模型决策解释。此外,数据还包括向用户展示的解释可视化结果、主题分析结果、用户提出的改进建议,以及初始问卷数据以评估参与者的领域知识与数据分析素养。所有转录文本均经人工标注,便于与相关片段自动匹配。随着XAI技术快速发展,跨学科定性评估解释能力成为重要议题。本数据集不仅可复现本研究,还为后续材料分析提供了广泛可能性。

原文摘要 · Abstract (English)

This paper introduces a dataset that is the result of a user study on the comprehensibility of explainable artificial intelligence (XAI) algorithms. The study participants were recruited from 149 candidates to form three groups representing experts in the domain of mycology (DE), students with a data science and visualization background (IT) and students from social sciences and humanities (SSH). The main part of the dataset contains 39 transcripts of interviews during which participants were asked to complete a series of tasks and questions related to the interpretation of explanations of decisions of a machine learning model trained to distinguish between edible and inedible mushrooms. The transcripts were complemented with additional data that includes visualizations of explanations presented to the user, results from thematic analysis, recommendations of improvements of explanations provided by the participants, and the initial survey results that allow to determine the domain knowledge of the participant and data analysis literacy. The transcripts were manually tagged to allow for automatic matching between the text and other data related to particular fragments. In the advent of the area of rapid development of XAI techniques, the need for a multidisciplinary qualitative evaluation of explainability is one of the emerging topics in the community. Our dataset allows not only to reproduce the study we conducted, but also to open a wide range of possibilities for the analysis of the material we gathered.

可解释AI用户研究数据集

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。