让大模型生成代码时能说清依据，提升开发者信任度。
Toward Explaining Large Language Models in Software Engineering Tasks
- 用沙普利值分析输入特征重要性，自动解释模型决策过程。
- 在代码生成与摘要任务中，解释准确率高于基线方法。
- 适合关注模型可信度的软件工程师和安全关键领域从业者。
大语言模型（LLMs）在软件工程（SE）自动化任务中取得显著进展，支持代码生成、代码摘要等复杂操作。然而，其黑箱特性阻碍了在高风险、安全关键场景中的应用，因可解释性与透明性对建立信任、责任追溯和有效人机协作至关重要。尽管可解释人工智能在软件工程领域受关注，现有方法缺乏与开发者实际推理逻辑一致的领域特定解释。为此，本文提出FeatureSHAP——首个完全自动化、模型无关的可解释性框架，专为软件工程任务设计。基于沙普利值，通过系统性输入扰动与任务特定相似性比较，将模型输出归因于高层次输入特征，兼容开源与私有LLM。我们在双模态SE任务（代码生成与代码摘要）上评估该方法，结果表明：FeatureSHAP对无关输入特征赋予更低重要性，解释保真度优于基线方法；37名实践者参与的调查证实，该方法有助于开发者更准确理解模型输出并做出更优决策。整体而言，FeatureSHAP为实现实用可解释人工智能迈出关键一步。代码已开源：https://github.com/deviserlab/FeatureSHAP。
原文摘要 · Abstract (English)
Recent progress in Large Language Models (LLMs) has substantially advanced the automation of software engineering (SE) tasks, enabling complex activities such as code generation and code summarization. However, the black-box nature of LLMs remains a major barrier to their adoption in high-stakes and safety-critical domains, where explainability and transparency are vital for trust, accountability, and effective human supervision. Despite increasing interest in explainable AI for software engineering, existing methods lack domain-specific explanations aligned with how practitioners reason about SE artifacts. To address this gap, we introduce FeatureSHAP, the first fully automated, model-agnostic explainability framework tailored to software engineering tasks. Based on Shapley values, FeatureSHAP attributes model outputs to high-level input features through systematic input perturbation and task-specific similarity comparisons, while remaining compatible with both open-source and proprietary LLMs. We evaluate FeatureSHAP on two bi-modal SE tasks: code generation and code summarization. The results show that FeatureSHAP assigns less importance to irrelevant input features and produces explanations with higher fidelity than baseline methods. A practitioner survey involving 37 participants shows that FeatureSHAP helps practitioners better interpret model outputs and make more informed decisions. Collectively, FeatureSHAP represents a meaningful step toward practical explainable AI in software engineering. FeatureSHAP is available at https://github.com/deviserlab/FeatureSHAP.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。