无需标注数据，用注意力机制精准定位代码缺陷
Where's the Bug? Attention Probing for Scalable Fault Localization
- 通过分析模型注意力分布实现无监督故障定位
- 在8个数据集上比最强基线提升34.6%精确率
- 效率远超大模型提示，适合实际工程部署
确保代码正确性仍是挑战，即便大型语言模型（LLMs）在代码任务中日益强大。尽管基于LLM的程序修复系统能仅凭用户报告提出修复方案，其效果仍受限于故障定位（FL）能力。现有方法依赖可执行测试用例，需昂贵且常有噪声的行级标注，或要求资源密集型的LLM。本文提出Bug Attention Probe（BAP），一种无需直接定位标签即可达到顶尖故障定位性能的方法，优于传统基线及大规模LLM提示。我们在多种代码场景下评估该方法，包括标准Defects4J数据集中的真实Java缺陷，以及另外七个涵盖多样缺陷类型和编程语言的数据集。在全部八个数据集上平均，BAP相比最强基线提升34.6%的top-1准确率，较零样本提示GPT-4o提升93.4%。BAP还显著更高效，以极小计算成本超越大型开源模型。
原文摘要 · Abstract (English)
Ensuring code correctness remains a challenging problem even as large language models (LLMs) become increasingly capable at code-related tasks. While LLM-based program repair systems can propose bug fixes using only a user's bug report, their effectiveness is fundamentally limited by their ability to perform fault localization (FL), a challenging problem for both humans and LLMs. Existing FL approaches rely on executable test cases, require training on costly and often noisy line-level annotations, or demand resource-intensive LLMs. In this paper, we present Bug Attention Probe (BAP), a method which learns state-of-the-art fault localization without any direct localization labels, outperforming traditional FL baselines and prompting of large-scale LLMs. We evaluate our approach across a variety of code settings, including real-world Java bugs from the standard Defects4J dataset as well as seven other datasets which span a diverse set of bug types and languages. Averaged across all eight datasets, BAP improves by 34.6% top-1 accuracy compared to the strongest baseline and 93.4% over zero-shot prompting GPT-4o. BAP is also significantly more efficient than prompting, outperforming large open-weight models at a small fraction of the computational cost.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。