系统研究代码混淆对大模型漏洞检测的影响,揭示其增效与失效场景。
A Systematic Study of Code Obfuscation Against LLM-based Vulnerability Detection
- 按布局、数据流、控制流分类11类19种混淆技术,统一框架评估
- 15个大模型+2个编程助手在4语言上测试,发现混淆既提升又降低检测率
- 结果受漏洞类型、代码特征和模型能力影响,为提升鲁棒性提供方向
随着大语言模型(LLMs)在代码漏洞检测中的广泛应用,其在多种漏洞类型下的可靠性与鲁棒性成为紧迫问题。传统对抗场景中,代码混淆长期被用作绕过审计工具的通用策略,在不修改工具本身的前提下保留漏洞可利用性。尽管已有众多研究探索混淆方法与工具,但其支持的技术、粒度和编程语言存在差异,难以系统评估其对基于LLM的漏洞检测的影响。为此,本文构建了混淆技术的系统化分类体系,涵盖三大类(布局、数据流、控制流)、11个子类及19种具体技术。我们采用一致的LLM驱动方法,在Solidity、C、C++和Python四种语言上实现这些技术,并评估其对15个主流大模型(覆盖DeepSeek、OpenAI、Qwen、LLaMA四个模型族)以及两个编程代理(GitHub Copilot和Codex)的影响。结果表明,代码混淆对基于LLM的漏洞检测既有正面也有负面影响,揭示了在特定条件下混淆可提升或削弱检测性能。进一步分析结合漏洞特征、代码属性与模型特性,指出若干开放问题,并提出未来增强真实场景下大模型鲁棒性的方向。
原文摘要 · Abstract (English)
As large language models (LLMs) are increasingly adopted for code vulnerability detection, their reliability and robustness across diverse vulnerability types have become a pressing concern. In traditional adversarial settings, code obfuscation has long been used as a general strategy to bypass auditing tools, preserving exploitability without tampering with the tools themselves. Numerous efforts have explored obfuscation methods and tools, yet their capabilities differ in terms of supported techniques, granularity, and programming languages, making it difficult to systematically assess their impact on LLM-based vulnerability detection. To address this gap, we provide a structured systematization of obfuscation techniques and evaluate them under a unified framework. Specifically, we categorize existing obfuscation methods into three major classes (layout, data flow, and control flow) covering 11 subcategories and 19 concrete techniques. We implement these techniques across four programming languages (Solidity, C, C++, and Python) using a consistent LLM-driven approach, and evaluate their effects on 15 LLMs spanning four model families (DeepSeek, OpenAI, Qwen, and LLaMA), as well as on two coding agents (GitHub Copilot and Codex). Our findings reveal both positive and negative impacts of code obfuscation on LLM-based vulnerability detection, highlighting conditions under which obfuscation leads to performance improvements or degradations. We further analyze these outcomes with respect to vulnerability characteristics, code properties, and model attributes. Finally, we outline several open problems and propose future directions to enhance the robustness of LLMs for real-world vulnerability detection.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。