用法庭辩论方式让AI agents检测代码漏洞，效果翻倍。
Let the Trial Begin: A Mock-Court Approach to Vulnerability Detection using LLM-Based Agents
- 四角色AI模拟法庭辩论，分别扮演安全研究员、作者、主持和评审。
- 相比最优基线，漏洞检测准确率几乎提升一倍，能发现真实世界中的零日漏洞。
- 小数据微调即可显著提升效果，且支持开源模型部署，解释质量高。
代码漏洞检测仍是关键挑战，尤其当正常函数与漏洞函数高度相似时。本文提出VulTrial——一种受法庭审判启发的多智能体框架，通过四个角色代理（安全研究员、代码作者、主持人、评审团）协同工作，利用GPT-4o作为基础大模型，使漏洞检测效能接近翻倍，优于现有最佳基线。此外，仅需少量数据进行角色专属指令微调，即可进一步提升性能。大量实验表明，VulTrial在多种大模型上表现优异，包括可本地部署的开源模型LLaMA-3.1-8B；生成的解释质量高，并成功识别出多个真实环境中存在的已确认零日漏洞。
原文摘要 · Abstract (English)
Detecting vulnerabilities in source code remains a critical yet challenging task, especially when benign and vulnerable functions share significant similarities. In this work, we introduce VulTrial, a courtroom-inspired multi-agent framework designed to identify vulnerable code and to provide explanations. It employs four role-specific agents, which are security researcher, code author, moderator, and review board. Using GPT-4o as the base LLM, VulTrial almost doubles the efficacy of prior best-performing baselines. Additionally, we show that role-specific instruction tuning with small quantities of data significantly further boosts VulTrial's efficacy. Our extensive experiments demonstrate the efficacy of VulTrial across different LLMs, including an open-source, in-house-deployable model (LLaMA-3.1-8B), as well as the high quality of its generated explanations and its ability to uncover multiple confirmed zero-day vulnerabilities in the wild.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。