arXiv:2501.04510cs.SEcs.AI2025-01中稿 · IEEE Transactions …被引 9

让大模型更懂代码结构,提升漏洞检测准确率

CGP-Tuning: Structure-Aware Soft Prompt Tuning for Code Vulnerability Detection

  • 用类型感知嵌入捕捉代码图的语义信息
  • 跨模态对齐模块线性计算,兼顾效率与效果
  • 适配多种代码大模型,推理快且精度高

大型语言模型(LLMs)被广泛用于软件漏洞检测,通常通过任务特定微调注入漏洞知识。然而,现有微调方法将源代码视为纯文本,丢失了代码固有的图结构信息。图增强的软提示微调试图将结构信息转化为模型可理解的上下文线索,但当前方法主要面向通用图任务,侧重邻接关系,难以保留代码图中的丰富语义信息(如控制流、数据流),且在跨模态对齐模块中难以兼顾计算效率。本文提出CGP-Tuning,一种面向漏洞检测的代码图增强型、结构感知的软提示微调方法。该方法引入类型感知嵌入以捕捉代码图中的丰富语义,并设计高效跨模态对齐模块,在保持线性计算复杂度的同时实现图-文交互。在最新DiverseVul数据集上,结合CodeLlama、CodeGemma和Qwen2.5-Coder三款开源代码LLM进行评估,结果表明CGP-Tuning实现模型无关提升,推理速度实用,平均比最优图增强基线高出4个百分点,较非微调零样本提示提升15个百分点。

原文摘要 · Abstract (English)

Large language models (LLMs) have been proposed as powerful tools for detecting software vulnerabilities, where task-specific fine-tuning is typically employed to provide vulnerability-specific knowledge to the LLMs. However, existing fine-tuning techniques often treat source code as plain text, losing the graph-based structural information inherent in code. Graph-enhanced soft prompt tuning addresses this by translating the structural information into contextual cues that the LLM can understand. However, current methods are primarily designed for general graph-related tasks and focus more on adjacency information, they fall short in preserving the rich semantic information (e.g., control/data flow) within code graphs. They also fail to ensure computational efficiency while capturing graph-text interactions in their cross-modal alignment module. This paper presents CGP-Tuning, a new code graph-enhanced, structure-aware soft prompt tuning method for vulnerability detection. CGP-Tuning introduces type-aware embeddings to capture the rich semantic information within code graphs, along with an efficient cross-modal alignment module that achieves linear computational costs while incorporating graph-text interactions. It is evaluated on the latest DiverseVul dataset and three advanced open-source code LLMs, CodeLlama, CodeGemma, and Qwen2.5-Coder. Experimental results show that CGP-Tuning delivers model-agnostic improvements and maintains practical inference speed, surpassing the best graph-enhanced soft prompt tuning baseline by an average of four percentage points and outperforming non-tuned zero-shot prompting by 15 percentage points.

漏洞检测代码大模型软提示微调图结构

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。