用大模型内部激活值提前检测代码漏洞，无需额外训练。
Probing the Prefill: Detecting Code Vulnerabilities via Latent Activations
- 从大模型预填充阶段提取代码激活值，用小型MLP探测漏洞信号。
- 在Devign上达68.8% F1，接近最优分类器性能。
- 仅需不到0.2%模型参数量，适合集成到生成流程中实时筛查。
基于大语言模型的代码生成已广泛应用于关键流程，但现有防御多为事后检测——依赖静态分析器、微调分类器或大模型评分器，在生成完成后才筛查代码漏洞，忽略了生成模型自身的内部状态。本文提出一个更聚焦、可直接测量的问题：当大模型将一段C/C++代码作为上下文读取时，其隐藏激活值是否已携带该代码漏洞状态的信号？我们从四个大模型（Granite-4.1-8B、Qwen3.5-9B、Qwen3.6-27B、Gemma-4-12B）中提取最后预填充标记的激活值，针对三个模型族训练小型MLP探测器，并在四个函数级C/C++漏洞检测基准（Devign、Big-Vul、Draper VDISC、PrimeVul）上评估。探测器平均F1达41.7%，使用1340万至1600万参数，不足基础模型规模的0.2%。在Devign上，最佳探测器（Qwen3.5-9B）达到68.8% F1，与公开的微调分类器最优结果（67.9%）相当，且仅读取冻结的通用大模型激活值；而在更难、更不平衡的基准（Big-Vul、Draper VDISC、PrimeVul）上，探测器表现仍显著落后于最优方法。这表明编码大模型对任意代码的表征已包含漏洞信息，为轻量级、模型原生的漏洞筛查提供了早期证据。
原文摘要 · Abstract (English)
LLM-based code generation is now embedded in mission-critical pipelines, but defenses against vulnerable output remain post-hoc -- static analyzers, fine-tuned classifiers, or an LLM judge that screen completed code, ignoring the generating model's own internal state. We test a narrower, directly measurable question: when an LLM reads a piece of C/C++ code as context, do its hidden activations already carry a signal about that code's vulnerability status? We extract last prefill token activations from four LLMs (Granite-4.1-8B, Qwen3.5-9B, Qwen3.6-27B, Gemma-4-12B) across three model families and train MLP probes on these activations. We evaluate them on four function-level C/C++ benchmarks (Devign, Big-Vul, Draper VDISC, PrimeVul). Our probes achieve 41.7\% average F1 using 13.4--16.0M-parameter probes -- under 0.2\% of base-model size. On Devign, the best probe (Qwen3.5-9B, 68.8\% F1) matches the published fine-tuned-classifier SOTA (67.9\%) despite reading only a frozen, general-purpose LLM's activations; on the harder, more imbalanced benchmarks (Big-Vul, Draper VDISC, PrimeVul) probes trail SOTA substantially. This is early evidence that a coding LLM's own representation of arbitrary code is informative about that code's vulnerability status, motivating further work toward lightweight, model-native vulnerability screening.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。