提出HarmAlign方法,在不破坏正常适配的前提下,有效防止模型被恶意微调。
Distribution-Specific Curvature Control with Finite-Sample Guarantees for Open-Weight Safety

- 基于对比激活子空间进行函数保持的谱变形,精准控制有害分布曲率。
- 在有限样本下证明了曲率下界与子空间能量的稳定性,保障安全约束。
- 可抵御多种攻击方式,且对正常任务训练无负面影响,适合高风险场景应用。
一次简短的微调可能瓦解开放权重模型的安全防护——重新训练一个拒绝有害内容的助手以协助武器开发或生成仇恨言论。如何在保留良性适应能力的同时防止此类有害微调仍具挑战:现有唯一具备显式曲率认证的方法(谱形变)会全局放大曲率,从而同时阻碍良性与有害适应。本文提出HarmAlign,沿估计的对比激活子空间实施函数保持的谱形变,并推导出估计子空间能量与局部有害分布曲率下界的有限样本边界。常步长梯度下降的稳定-进展二分法将认证曲率转化为条件收敛速率控制。实证表明,在固定架构、有限预算的一阶威胁模型下,HarmAlign能阻断直接微调及三种数据或目标自适应攻击,覆盖危险知识重学习和有害协助微调场景,而良性任务仍可正常训练。该保护效果在所有一阶优化器变体与各攻击检查点均持续存在,且适用于意外安全退化与涌现错对齐等重要情形。
原文摘要 · Abstract (English)
A short fine-tuning run can undo the safety guards of an open-weight model---retraining a refusal-trained assistant to aid weapons development or produce hate speech. Preventing such harmful fine-tuning while retaining benign adaptability remains difficult: the only prior method with an explicit curvature certificate, spectral deformation, inflates curvature globally and thereby obstructs benign adaptation along with harmful adaptation. We propose HarmAlign, which applies function-preserving spectral deformation along a estimated contrastive activation subspace. We derive finite-sample bounds for the estimated subspace energy and the resulting local harmful-distribution curvature lower bound. A stability--progress dichotomy for constant-step gradient descent turns the certified curvature into conditional convergence-rate control. Empirically, within a fixed-architecture, finite-budget first-order threat model, HarmAlign blocks direct fine-tuning and three data- or objective-adaptive attacks across a hazardous-knowledge relearning setting and a harmful-assistance fine-tuning setting, while the protected benign tasks remain trainable. The block persists across the tested first-order optimizer variants over every attack checkpoint, and under out-of-distribution harmful fine-tuning, and it extends to important cases in our threat model: accidental safety degradation and emergent misalignment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。