测试了代码改写绕过大模型漏洞检测的效果，发现多数检测器极易被绕过。
Syntax- and Compilation-Preserving Evasion of LLM Vulnerability Detectors
- 用语法和编译保持不变的代码改写攻击五种变体
- 70%以上准确率的模型仍有超87%漏洞可被绕过
- 通用对抗字符串可跨模型攻击，适合安全评估者参考
基于大模型的漏洞检测器日益用于CI/CD安全防护，但其在语法与编译保持不变的代码改写攻击下的鲁棒性仍不明确。我们在统一的C/C++基准（N=5000）上评估了五种攻击变体，涵盖四类行为保持型代码变换，并提出完全抗性（CR）指标，衡量正确检测到的漏洞中能抵御所有攻击变体的比例。结果揭示显著的鲁棒性差距：清洁召回率超过70%的模型，其CR低至0.12%，意味着超过87%的已检出漏洞可被至少一种语法保持型改写绕过。在140亿参数的替代模型上优化的通用对抗字符串，可有效迁移至GPT-4o等黑盒API；而针对目标模型的优化进一步提升绕过率（最高达92.5%的攻击成功率）。这表明，仅依赖清洁基准准确率无法作为部署中漏洞检测器的安全保障。
原文摘要 · Abstract (English)
LLM-based vulnerability detectors are increasingly deployed in CI/CD security gating, yet their resilience to evasion under syntax- and compilation-preserving edits remains poorly understood. We evaluate five attack variants spanning four carrier families of behavior-preserving code transformations on a unified C/C++ benchmark ($N=5000$) and introduce Complete Resistance (CR), measuring the fraction of correctly detected vulnerabilities that withstand all attack variants. Our findings reveal a significant robustness gap: models achieving 70\%+ clean recall exhibit CR as low as 0.12\%, meaning over 87\% of detected vulnerabilities can be evaded by at least one syntax-preserving edit. Universal adversarial strings optimized on a 14B surrogate transfer effectively to black-box APIs including GPT-4o, while on-target optimization further amplifies evasion (up to 92.5\% ASR). These results indicate that clean benchmark accuracy alone is insufficient as a security guarantee for deployed vulnerability detectors.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。