通过修正模型内部表示，让大模型生成更安全的代码。
A Mixture of Linear Corrections Generates Secure Code
- 用线性修正融合方式调节模型生成概率，引导输出更安全代码。
- 使Qwen2.5-Coder-7B的安全率提升8.9%，功能准确率提高2.1%。
- 无需训练即可在推理时增强代码安全性，适合代码生成场景使用。
大语言模型在代码生成任务中已表现卓越，但难以可靠检测或避免代码漏洞。这种缺陷是由于对漏洞学习不足，还是提示方式无效？我们通过表征工程方法，研究大模型是否内含识别漏洞所需的概念。结果发现，当前模型已具备区分漏洞与安全代码的精确内部表征，其效果优于标准提示方法。基于这些敏感于漏洞的表征，我们提出一种推理时调优技术——混合修正（MoC），通过一系列线性修正微调模型的词元生成概率。该方法有效引导模型生成漏洞更少的代码，且不损害功能表现，在不改变模型参数的前提下实现了可控的漏洞管理。显著地，该方法使Qwen2.5-Coder-7B的安全率提升8.9%，同时在HumanEval上的通过率（pass@1）提升2.1%。
原文摘要 · Abstract (English)
Large language models (LLMs) have become proficient at sophisticated code-generation tasks, yet remain ineffective at reliably detecting or avoiding code vulnerabilities. Does this deficiency stem from insufficient learning about code vulnerabilities, or is it merely a result of ineffective prompting? Using representation engineering techniques, we investigate whether LLMs internally encode the concepts necessary to identify code vulnerabilities. We find that current LLMs encode precise internal representations that distinguish vulnerable from secure code--achieving greater accuracy than standard prompting approaches. Leveraging these vulnerability-sensitive representations, we develop an inference-time steering technique that subtly modulates the model's token-generation probabilities through a mixture of corrections (MoC). Our method effectively guides LLMs to produce less vulnerable code without compromising functionality, demonstrating a practical approach to controlled vulnerability management in generated code. Notably, MoC enhances the security ratio of Qwen2.5-Coder-7B by 8.9\%, while simultaneously improving functionality on HumanEval pass@1 by 2.1\%.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。