用对抗式多智能体机制，大幅降低大模型缺陷发现中的误报率。
Refute-or-Promote: An Adversarial Stage-Gated Multi-Agent Review Methodology for High-Precision LLM-Assisted Defect Discovery
- 通过分层上下文搜索与跨模型批判，构建多阶段验证流程
- 在7个目标中淘汰79%的假阳性候选，实测杀灭率达83%
- 适合需要高可信缺陷报告的安全研究者和开源维护团队
大模型辅助缺陷发现面临精度危机：大量看似合理实则错误的报告严重干扰维护者工作并损害真实发现的可信度。本文提出Refute-or-Promote方法，结合分层上下文搜寻（SCH）、对抗性否定指令、上下文不对称性与跨模型批评者（CMC），在每个晋升阶段部署对抗智能体主动质疑候选结果；冷启动评审可减少锚定效应；跨家族审查能发现同家族审查遗漏的相关盲区。在为期31天、覆盖7个目标（安全库、ISO C++标准、主流编译器）的实验中，该流程在回溯汇总中淘汰了171个候选中的约79%；在精简协议子集（lcms2、wolfSSL；n=30）上，前瞻性杀灭率达到83%。成果包括4个CVE（3个公开，1个保密）、LWG 4549被纳入C++工作草案、5个已合并的编辑修正补丁、3个编译器一致性缺陷、8个未发CVE的安全修复、1项RFC 9000勘误提交委员会评审，以及1个以上符合FIPS 140-3规范的问题正在协调披露——均经外部认可而非基准测试验证。最典型失败案例：10名专职评审一致认可一个不存在于OpenSSL CMS模块的Bleichenbacher填充漏洞，最终仅由一次实测验证才被剔除，由此确立强制实测门禁。所有漏洞均非自主发现，贡献在于构建外部结构以过滤大模型持续存在的假阳性。作为初步迁移测试，简化版跨家族批判变体还解决了此前未解的5个SymPy实例（SWE-bench Verified）及1个SWE-rebench难题任务。
原文摘要 · Abstract (English)
LLM-assisted defect discovery has a precision crisis: plausible-but-wrong reports overwhelm maintainers and degrade credibility for real findings. We present Refute-or-Promote, an inference-time reliability pattern combining Stratified Context Hunting (SCH) for candidate generation, adversarial kill mandates, context asymmetry, and a Cross-Model Critic (CMC). Adversarial agents attempt to disprove candidates at each promotion gate; cold-start reviewers are intended to reduce anchoring cascades; cross-family review can catch correlated blind spots that same-family review misses. Over a 31-day campaign across 7 targets (security libraries, the ISO C++ standard, major compilers), the pipeline killed roughly 79% of 171 candidates before advancing to disclosure (retrospective aggregate); on a consolidated-protocol subset (lcms2, wolfSSL; n=30), the prospective kill rate was 83%. Outcomes: 4 CVEs (3 public, 1 embargoed); LWG 4549 accepted to the C++ working paper; 5 merged C++ editorial PRs; 3 compiler conformance bugs; 8 merged security-related fixes without CVE; an RFC 9000 errata filed under committee review; and 1+ FIPS 140-3 normative compliance issues under coordinated disclosure -- all evaluated by external acceptance, not benchmarks. The most instructive failure: ten dedicated reviewers unanimously endorsed a non-existent Bleichenbacher padding oracle in OpenSSL's CMS module; it was killed only by a single empirical test, motivating the mandatory empirical gate. No vulnerability was discovered autonomously; the contribution is external structure that filters LLM agents' persistent false positives. As a preliminary transfer test beyond defect discovery, a simplified cross-family critique variant also solved five previously unsolved SymPy instances on SWE-bench Verified and one SWE-rebench hard task.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。