通过联合约束权重与激活，防止大模型微调时安全能力退化。
Preventing Safety Drift in Large Language Models via Coupled Weight and Activation Constraints
- 联合限制模型权重更新和关键特征，提升安全稳定性。
- 在四种模型上测试，有害响应率显著降低，精度影响小。
- 适合需要高安全性的大模型微调场景，如医疗、金融。
大型语言模型（LLM）在微调过程中，安全对齐能力极易退化，即使良性调整也可能削弱预训练的拒绝能力并诱发有害响应。现有方法通常仅约束权重或激活，未考虑二者耦合效应。本文理论证明，单独约束任一变量均不足以保障安全。为此提出耦合权重与激活约束（CWAC）：一方面对权重更新施加预计算的安全子空间约束，另一方面利用稀疏自编码器识别关键安全特征并进行针对性正则化。在四个主流大模型及多种下游任务上的实验证明，CWAC始终以最低有害得分表现，且对微调精度影响极小，在高有害数据比例下仍显著优于强基线。
原文摘要 · Abstract (English)
Safety alignment in Large Language Models (LLMs) remains highly fragile during fine-tuning, where even benign adaptation can degrade pre-trained refusal behaviors and enable harmful responses. Existing defenses typically constrain either weights or activations in isolation, without considering their coupled effects on safety. In this paper, we first theoretically demonstrate that constraining either weights or activations alone is insufficient for safety preservation. To robustly preserve safety alignment, we propose Coupled Weight and Activation Constraints (CWAC), a novel approach that simultaneously enforces a precomputed safety subspace on weight updates and applies targeted regularization to safety-critical features identified by sparse autoencoders. Extensive experiments across four widely used LLMs and diverse downstream tasks show that CWAC consistently achieves the lowest harmful scores with minimal impact on fine-tuning accuracy, substantially outperforming strong baselines even under high harmful data ratios.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。