将代码代理的执行日志转化为可操作的故障解释，提升调试效率
XAI for Coding Agent Failures: Transforming Raw Execution Traces into Actionable Insights
- 构建领域专属故障分类体系，自动标注失败类型
- 用户定位故障原因快2.8倍，修复准确率提升73%
- 适合开发者和非技术人员快速理解复杂错误
基于大语言模型的代码代理在自动化开发任务中展现潜力，但其故障常难以理解和调试。尽管通用大模型如GPT能提供临时解释，原始执行日志对经验丰富的开发者仍难解读。本文提出系统性可解释AI（XAI）方法，将原始执行轨迹转化为结构化、人类可读的解释。方法包含三个核心组件：(1) 基于真实代理故障分析构建的领域专用故障分类体系；(2) 使用定义标注模式自动分类故障的标注系统；(3) 生成可视化执行流程、自然语言解释与可操作建议的混合解释器。通过20名参与者（10名技术人员，10名非技术人员）的用户研究发现，该方法使用户定位故障根因速度提升2.8倍，修复准确率提高73%，显著优于现有自适应解释模型。该结构化方法提供一致且领域相关的洞察，并集成可视化，为软件开发工作流中的可解释人工智能系统提供了系统性框架。
原文摘要 · Abstract (English)
Large Language Model (LLM)-based coding agents show promise in automating software development tasks, yet they frequently fail in ways that are difficult for developers to understand and debug. While general-purpose LLMs like GPT can provide ad-hoc explanations of failures, raw execution traces remain challenging to interpret even for experienced developers. We present a systematic explainable AI (XAI) approach that transforms raw agent execution traces into structured, human-interpretable explanations. Our method consists of three key components: (1) a domain-specific failure taxonomy derived from analyzing real agent failures, (2) an automatic annotation system that classifies failures using defined annotation schema, (3) a hybrid explanation generator that produces visual execution flows, natural language explanations, and actionable recommendations. Through a user study with 20 participants (10 technical, 10 non-technical), we demonstrate that our approach enables users to identify failure root causes 2.8 times faster and propose correct fixes with 73% higher accuracy compared to raw execution traces. Importantly, our structured approach outperforms ad-hoc state of the art models explanations by providing consistent, domain-specific insights with integrated visualizations. Our work establishes a framework for systematic agent failure analysis, addressing the critical need for interpretable AI systems in software development workflows
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。