研究如何安全地移除大模型的安全对齐,避免误拒合法安全请求。
Ablating Safety: Mechanisms for Removing Alignment in Language Models for Security Applications
- 通过投影和LoRA微调等方法可控移除模型对齐机制
- 任务专用微调使安全任务成功率提升至87%,但需警惕副作用
- 强调不能仅凭合规性判断模型是否可用,需综合评估风险与能力
安全对齐的语言模型常因关键词相似而拒绝合法的网络安全请求,导致安全评估模糊:失败可能源于能力不足或政策拦截。本文提出‘去对齐’实验框架,系统测试授权提示、可逆拒绝方向投影、表示控制投影及基于LoRA的去对齐/任务适配方法。在包含60个提示的Security-AR测评集上,单向量拒绝投影仅将平均安全得分从0.46提升至0.50,同时不安全合规率从0.10升至0.47;秩4拒绝子空间投影达0.51且保持原生溢出率。任务专用LoRA将安全得分提至0.87(通用能力0.83,不安全合规0.13),而拒绝抑制保留则使溢出率升至0.27。结果表明,去对齐应视为效用-风险权衡,而非简单去屏蔽,且合规性本身不能代表能力或安全部署。
原文摘要 · Abstract (English)
Safety-aligned language models often refuse cybersecurity requests whose wording resembles misuse, even when the task is authorized and defensive. This makes security evaluation ambiguous: a failed answer may reflect missing capability or refusal-policy intervention. Ablating Safety studies alignment removal as a controlled transformation-evaluation protocol for authorized security tasks, comparing authorized-context prompting, reversible refusal-direction activation projection, representation-control projections, and LoRA-based de-alignment or task adaptation. We evaluate refusal, attempt rate, validated security success, general-capability retention, instability, and out-of-scope unsafe compliance on Security-AR, a 60-prompt suite of authorized security, benign general, and non-operational spillover probes. The reported runs include a four-model projection pilot with 416 completions, a three-model Qwen2.5 LoRA extension with 1,980 held-out completions, representation and robustness sweeps, and executable secure-repair validators. Single-vector refusal projection raises mean security score only from 0.46 to 0.50 while increasing unsafe compliance from 0.10 to 0.47; rank-4 refusal-subspace projection reaches 0.51 while matching the aligned spillover rate. Task-only LoRA raises mean security score to 0.87 with general score 0.83 and unsafe compliance 0.13, while refusal-suppression with retention raises spillover to 0.27. These results support evaluating alignment removal as a utility-risk frontier, not as an uncensoring recipe, and treating compliance alone as neither competence nor safe deployment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。