通过权重编辑解除代码大模型的拒绝能力，实现可控漏洞生成。
Willing but Unable: Separating Refusal from Capability in Code LLMs via Abliteration
- 用低秩权重编辑移除模型拒绝注入漏洞的倾向，保留生成能力。
- 编辑后所有模型拒绝率降至零，语法正确率超93%。
- 适合研究漏洞生成、安全对齐与模型可操控性的研究人员。
基于学习的漏洞检测面临标注脆弱代码规模化难题：挖掘语料存在大量标签噪声，现有大模型增强方法因改造已有脆弱代码而传播错误。另一路径是从安全代码出发，让指令微调的大模型按指定CWE注入漏洞（将标签负担从开放检测转为有限二元确认），但安全对齐的代码大模型系统性拒绝此类提示。本文首次探索“消解”（abliteration）——一种在残差流中正交投影去除拒绝方向的低秩权重编辑技术，作为突破该障碍的工具。以Python和CWE-89（SQL注入）为例，评估Qwen2.5-Coder-Instruct系列（3B、7B、14B）在PromSec与SafeCoder抽取的安全样本上，每条件重复三次。结果表明：（i）拒绝行为强烈依赖模型规模与提示上下文——14B拒绝100%，7B在PromSec拒绝73%但仅5%在SafeCoder，3B基本不被阻断；（ii）消解后各规模模型拒绝率降至零或接近零，同时语法正确率保持在93%以上，支持拒绝可与生成能力分离的观点；（iii）消解后注入率仍受模型容量限制（14B为88-97%，7B为89-90%，3B为25-48%），区分了意愿（经消解解锁）与能力（随参数增长）。漏洞判定由CodeQL、Semgrep、Bandit三工具集成检测，并经两名作者人工复核阳性输出。
原文摘要 · Abstract (English)
Producing a labeled vulnerable code at scale is a recurring obstacle for learning-based vulnerability detection: mined corpora carry substantial label noise, and existing LLM-based augmentation propagates these inaccuracies because it transforms vulnerable seeds rather than synthesising vulnerabilities from a specification. A complementary route is to start from safe code and ask an instruction-tuned LLM to inject a specified CWE (which would shift the labeling burden from open-ended detection to bounded binary confirmation) but safety-aligned code LLMs systematically refuse such prompts. This paper is a preliminary feasibility study of abliteration, a low-rank weight edit that orthogonally projects out the refusal direction in the residual stream, as a tool to remove this barrier. We use Python and CWE-89 (SQL injection) as a case study, evaluating the Qwen2.5-Coder-Instruct family at 3B, 7B, and 14B parameters on safe samples drawn from PromSec and SafeCoder, replicated three times per condition. We find that (i) refusal on injection prompts is strongly size- and prompt-context-dependent: the 14B refuses 100% of prompts, the 7B refuses 73% of PromSec but only 5% of SafeCoder, whereas the 3B is essentially never blocked; (ii) abliteration reduces refusal to zero or near-zero across all sizes while leaving syntactic validity above 93%, supporting the view that, in this setting, refusal can be detached from measured code-generation capability; and (iii) the post-abliteration injection rate remains capacity-bound (88-97% on the 14B, 89-90% on the 7B, and 25-48% on the 3B) separating willingness, which abliteration unlocks, from capability, which scales with parameters. Vulnerability verdicts are produced by a three-tool detector ensemble (CodeQL, Semgrep, Bandit) followed by manual adjudication by two authors on detector-positive outputs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。