防止大模型持续适应中安全性能逐步退化，保持核心安全能力不变。
SafeAnchor: Preventing Cumulative Safety Erosion in Continual Domain Adaptation of Large Language Models
- 通过分析参数空间识别安全子空间，限制更新方向
- 在多领域连续微调中维持93.2%原始安全水平
- 适合需要长期安全稳定的部署场景
大语言模型的安全对齐极为脆弱：安全特性集中于前几个输出词元，仅需100个对抗样本微调即可逆转。在真实部署中，模型需依次适应医学、法律、编程等多领域，导致安全防护能力逐步退化。现有方法仅针对单任务微调，无法应对多领域连续适应场景。本文提出SafeAnchor框架，在持续适应过程中锚定安全能力。首先通过费舍尔信息特征分解在LoRA参数空间中识别低秩安全子空间；随后将领域特定梯度更新约束在该子空间的正交补空间内；最后通过阈值触发的纠正性重放监控残余安全漂移。在Llama-2-7B-Chat与Mistral-7B-Instruct上，经三领域流程和八个基准测试验证，SafeAnchor保留了93.2%的原始安全对齐，优于所有基线18-42个百分点，同时在领域任务表现上仅比无约束微调低1.5个百分点。
原文摘要 · Abstract (English)
Safety alignment in large language models is remarkably shallow: it is concentrated in the first few output tokens and reversible by fine-tuning on as few as 100 adversarial examples. This fragility becomes critical in real-world deployment, where models undergo sequential adaptation across domains such as medicine, law, and code, causing safety guardrails to erode cumulatively. Yet all existing safety-preserving methods target only single-task fine-tuning, leaving the multi-domain sequential setting entirely unaddressed. We introduce SafeAnchor, a framework that anchors safety in place throughout continual adaptation. SafeAnchor first identifies low-rank safety subspaces in LoRA parameter space via Fisher Information eigendecomposition, then constrains domain-specific gradient updates to the orthogonal complement of these subspaces, and finally monitors for residual safety drift with threshold-triggered corrective replay. Evaluated on Llama-2-7B-Chat and Mistral-7B-Instruct across a three-domain pipeline and eight benchmarks, SafeAnchor retains 93.2% of original safety alignment, outperforming all baselines by 18-42 points, while matching unconstrained fine-tuning to within 1.5 points on domain tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。