让AI修代码时记住失败经验，提升修复成功率。
FailureMem: A Failure-Aware Multimodal Framework for Autonomous Software Repair
- 用混合工作流+智能感知，精准定位界面问题区域。
- 在SWE-bench多模态数据集上修复率比GUIRepair高3.7%。
- 适合需要自动修复带界面缺陷的软件项目开发者。
多模态自动化程序修复（MAPR）要求模型同时理解源代码、文本问题描述和图形界面截图。尽管基于大模型的修复系统表现良好，但现有方法存在流程僵化、视觉推理缺乏局部定位、失败尝试难以复用等问题。为此，我们提出FailureMem框架，包含三项核心技术：融合结构化定位与灵活推理的混合工作流代理架构，支持区域级视觉定位的主动感知工具，以及将过往修复尝试转化为可复用知识的失败记忆库。在SWE-bench Multimodal数据集上的实验表明，FailureMem相较于GUIRepair的修复成功率提升了3.7%。
原文摘要 · Abstract (English)
Multimodal Automated Program Repair (MAPR) extends traditional program repair by requiring models to jointly reason over source code, textual issue descriptions, and visual artifacts such as GUI screenshots. While recent LLM-based repair systems have shown promising results, existing approaches face several limitations: rigid workflow pipelines restrict exploration during debugging, visual reasoning is often performed over full-page screenshots without localized grounding, and failed repair attempts are rarely transformed into reusable knowledge. To address these challenges, we propose FailureMem, a multimodal repair framework that integrates three key mechanisms: a hybrid workflow-agent architecture that balances structured localization with flexible reasoning, active perception tools that enable region-level visual grounding, and a Failure Memory Bank that converts past repair attempts into reusable guidance. Experiments on SWE-bench Multimodal demonstrate FailureMem improves the resolved rate over GUIRepair by 3.7%.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。