arXiv:2505.14185cs.LGcs.AI2025-05被引 7

安全行为与通用能力深度纠缠,无法通过线性子空间分离

Safety Subspaces are Not Linearly Distinct: A Fine-Tuning Case Study

  • 分析权重与激活空间,发现安全行为与有用能力共用同一子空间
  • 不同安全程度的提示激活重叠的神经表示,无明显区分模式
  • 挑战子空间防御思路,提示后续训练需新策略

大型语言模型依赖安全对齐生成符合社会规范的回复,但该行为易受破坏:即使在良性或轻度污染数据上微调,也可能导致安全性能下降并重现有害行为。已有研究认为对齐可能对应权重空间中可识别的方向,形成可隔离或保留的子空间以抵御偏差。本文开展全面实证研究,检验安全相关行为是否集中于特定线性子空间、能否与通用学习分离,以及有害性是否源于激活中的可区分模式。在权重和激活空间中,结果一致:增强安全性的子空间同时放大有用能力,不同安全含义的提示激活重叠的表示。安全并非位于独立方向,而是高度缠绕于模型的通用学习成分之中。这表明基于子空间的防御存在根本局限,亟需替代策略以维持持续训练下的安全性。我们在五个来自 Llama 与 Qwen 系列的开源大模型上进行了多组实验验证,代码已公开于 https://github.com/CERT-Lab/safety-subspaces。

原文摘要 · Abstract (English)

Large Language Models (LLMs) rely on safety alignment to produce socially acceptable responses. However, this behavior is known to be brittle: further fine-tuning, even on benign or lightly contaminated data, can degrade safety and reintroduce harmful behaviors. A growing body of work suggests that alignment may correspond to identifiable directions in weight space, forming subspaces that could, in principle, be isolated or preserved to defend against misalignment. In this work, we conduct a comprehensive empirical study of this perspective. We examine whether safety-relevant behavior is concentrated in specific linear subspaces, whether it can be separated from general-purpose learning, and whether harmfulness arises from distinguishable patterns in activations. Across both weight and activation spaces, our findings are consistent: subspaces that amplify safe behaviors also amplify useful ones, and prompts with different safety implications activate overlapping representations. Rather than residing in distinct directions, we show that safety is highly entangled with the general learning components of the model. This suggests that subspace-based defenses face fundamental limitations and underscores the need for alternative strategies to preserve safety under continued training. We corroborate these findings with multiple experiments on five open-source LLMs from the Llama and Qwen families. Our code is publicly available at: https://github.com/CERT-Lab/safety-subspaces.

安全对齐大模型微调风险子空间

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。