arXiv:2605.17561cs.SEcs.AI2026-05

自动分类无效漏洞报告并生成无需改代码的修复方案

Automated Root-Cause Subclassification and No-Code Fix Generation for Invalid Bug Reports

论文配图:Automated Root-Cause Subclassification and No-Code Fix Generation for Invalid Bug Reports
图 1 · 摘自论文原文
  • 构建标准化分类体系,用大模型分析无效报告原因
  • RAG方法在分类上表现最佳,加权F1达0.66;网页搜索在修复建议上成功率最高达68.9%
  • 适合软件支持团队和自动化运维工具开发者使用

软件使用问题以漏洞报告形式提交,但许多报告无效,仅需无代码修复。手动判断无效报告根因并提供解决方案造成资源浪费。本文提出面向根因的无效报告分类标准,构建金标准基准测试集,评估了纯大模型、检索增强生成(RAG)及代理式网页搜索在分类与无代码修复生成上的表现。通过人工标注的真值数据对比,分类采用加权F1评分,修复建议用BERTScore和裁判大模型成功率评估。结果显示,RAG在分类上表现最优(加权F1=0.66),优于纯大模型(0.65)和代理搜索(0.64)。子类中,不可复现(F1=0.85)和功能请求/疑问(F1=0.79)表现较好,错误版本仍最困难(F1 0.00–0.29)。修复生成方面,代理搜索成功率最高(68.9%),优于RAG(64.4%)和纯大模型(64.9%),其中“按设计工作”类达到87.4%峰值。

原文摘要 · Abstract (English)

Issues faced when using software are reported in the form of bug reports. However, many bug reports are invalid, meaning they do not require code changes, and are resolved with a no-code fix. Manually determining the root cause of the invalid bug reports and providing actionable resolutions by the customer support causes a serious waste of resources. Our goal is to introduce a standardized taxonomy for root-cause oriented invalid bug report subclassification, and perform experiments to test the accuracy of various approaches on invalid subclassification and no-code fix generation. We study how different configurations perform on a gold-standard benchmark we have created. Using a manually curated benchmark for higher quality analysis, we experimented with vanilla LLMs, Retrieval Augmented Generation, and agentic web search to identify invalid subclasses and generate no-code fixes. We evaluated the results against manually labeled ground truth data that includes the invalid subclass and no-code fixes from the original bug reports. We measured subclass detection performance with weighted F1-Score, and assessed no-code fix suggestions using BERTScore and Judge LLM success rates. For subclassification, retrieval augmented generation achieves the highest overall performance with 0.66 weighted F1, slightly outperforming vanilla LLMs at 0.65 and agentic web search at 0.64. At the subclass level, performance peaks at 0.85 F1 for Non-reproducibility and 0.79 for Feature Request and Question, while Wrong Version remains the most challenging with scores between 0.00 and 0.29. For no-code fix generation, agentic web search achieves the highest overall Judge LLM success rate at 68.9%, compared to 64.4% for RAG applications and 64.9% for vanilla LLMs, with subclass-level peaks of 87.4% for Working as Designed and 72.2% for Question.

缺陷报告大模型应用自动化修复RAG

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。