让大模型自动生成可运行的漏洞验证代码，提升缺陷检测的可信度与效率。
AnyPoC: Universal Proof-of-Concept Test Generation for Scalable LLM-Based Bug Detection
- 构建多智能体框架，通过执行验证和独立复核减少幻觉与误报。
- 在12个大型系统中生成37%更多有效漏洞验证代码，误报率降低9.7倍。
- 可发现121个新漏洞，其中108个被开发者确认，92个已修复。
尽管基于大模型的智能体能识别大量代码候选缺陷，但其报告仍为静态假设，需人工验证，限制了自动化检测的实用性。本文将此问题视为测试生成任务：给定候选报告，生成可执行的漏洞验证（PoC）——如脚本、命令序列或构造输入——以触发潜在缺陷。自动化PoC生成可作为可扩展的验证标准，实现端到端自主缺陷检测，提供具体的执行证据。然而，简单的大模型智能体不可靠：它们倾向“成功”结果，可能通过生成看似合理但无效的PoC或虚构执行轨迹进行奖励欺骗。为此，我们提出ANYPoC，一个通用的多智能体框架，包含三步：(1) 分析并事实核查候选漏洞报告；(2) 迭代合成并执行PoC，收集执行轨迹；(3) 独立重执行并严格审查PoC，以缓解幻觉与奖励欺骗。此外，ANYPoC持续提取并演化PoC知识库，以应对多样化任务。该框架可处理任意来源的漏洞报告，并兼容不同报告器。为验证实用性和通用性，我们将ANYPoC与一个简单代理式漏洞报告器结合，在12个大规模关键软件系统（包括Firefox、Chromium、LLVM、OpenSSL、SQLite、FFmpeg、Redis）上进行测试。相比当前最优编码代理（如Claude Code和Codex），ANYPoC对真实阳性漏洞报告生成37%更多的有效PoC，拒绝9.7倍更多的假阳性报告。此外，从两千多个噪声报告中发现121个新漏洞，其中108个经开发人员确认，92个已修复，46个被采纳为官方回归测试。
原文摘要 · Abstract (English)
While recent LLM-based agents can identify many candidate bugs in source code, their reports remain static hypotheses that require manual validation, limiting the practicality of automated bug detection. We frame this challenge as a test generation task: given a candidate report, synthesizing an executable proof-of-concept (PoC) - such as a script, command sequence, or crafted input - to trigger the suspected defect. Automated PoC generation can act as a scalable validation oracle, enabling end-to-end autonomous bug detection by providing concrete execution evidence. However, naive LLM agents are unreliable validators: they are biased toward "success" and may reward-hack by producing plausible but non-functional PoCs or even hallucinated traces. To address this, we present ANYPoC, a general multi-agent framework that (1) analyzes and fact-checks a candidate bug report, (2) iteratively synthesizes and executes a PoC while collecting execution traces, and (3) independently re-executes and scrutinizes the PoC to mitigate hallucination and reward hacking. In addition, ANYPoC also continuously extracts and evolves a PoC knowledge base to handle heterogeneous tasks. ANYPoC operates on candidate bug reports regardless of their source and can be paired with different bug reporters. To demonstrate practicality and generality, we apply ANYPoC, together with a simple agentic bug reporter, on 12 large-scale, critical software systems, including Firefox, Chromium, LLVM, OpenSSL, SQLite, FFmpeg, and Redis. Compared to the state-of-the-art coding agents, e.g., Claude Code and Codex, ANYPoC produces 37% more valid PoCs for true-positive bug reports and rejects 9.7x more false-positive bug reports. ANYPoC also enables the discovery of 121 new bugs from over two thousand noisy bug reports, with 108 confirmed by developers and 92 fixed. 46 PoCs have also been adopted as official regression tests.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。