提出分段解释方法,缓解文本摘要中AI解释不一致问题
"Let's Agree to Disagree": Investigating the Disagreement Problem in Explainable AI for Text Summarization
- 将文章分段后局部使用XAI方法生成解释
- 在Xsum和CNN/Daily Mail数据集上显著降低解释分歧
- 支持句级可视化,帮助用户理解模型决策依据
文本摘要中的可解释人工智能(XAI)对理解模型行为、建立信任至关重要。尽管XAI有效,但不同方法对同一结果常给出冲突解释,即‘解释分歧’问题,影响可信度与安全性。本文首次实证研究该问题,发现其广泛存在于主流摘要模型中。为此提出区域可解释AI(RXAI),利用句子嵌入与聚类将文章划分为连贯片段,对各片段分别应用XAI生成局部解释,以减少跨方法分歧。实验在极端摘要(Xsum)和CNN/Daily Mail数据集上验证,局部解释一致性显著提升。同时开发交互式JavaScript工具,支持句级颜色标注归因分数,增强用户对解释的理解。
原文摘要 · Abstract (English)
Explainable Artificial Intelligence (XAI) methods in text summarization are essential for understanding the model behavior and fostering trust in model-generated summaries. Despite the effectiveness of XAI methods, recent studies have highlighted a key challenge in this area known as the "disagreement problem". This problem occurs when different XAI methods yield conflicting explanations for the same model outcome. Such discrepancies raise concerns about the consistency of explanations and reduce confidence in model interpretations, which is crucial for secure and accountable AI applications. This work is among the first to empirically investigate the disagreement problem in text summarization, demonstrating that such discrepancies are widespread in state-of-the-art summarization models. To address this gap, we propose Regional Explainable AI (RXAI) a novel segmentation-based approach, where each article is divided into smaller, coherent segments using sentence transformers and clustering. We use XAI methods on text segments to create localized explanations that help reduce disagreement between different XAI methods, thereby enhancing the trustworthiness of AI-generated summaries. Our results illustrate that the localized explanations are more consistent than full-text explanations. The proposed approach is validated using two benchmark summarization datasets, Extreme summarization (Xsum) and CNN/Daily Mail, indicating a substantial decrease in disagreement. Additionally, the interactive JavaScript visualization tool is developed to facilitate easy, color-coded exploration of attribution scores at the sentence level, enhancing user comprehension of model explanations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。