arXiv:2605.30640cs.LGcs.CL2026-05被引 1

用安全方向投影修复LoRA微调,既防攻击又保效果。

CSULoRA: Closest Safe Update Low-Rank Adaptation

论文配图:CSULoRA: Closest Safe Update Low-Rank Adaptation
图 1 · 摘自论文原文
  • 通过权重偏移估计安全子空间,分解更新成分。
  • 对抗训练下攻击成功率大幅下降,保留大部分任务性能。
  • 无需额外训练,适合需安全性的模型微调场景。

低秩适配(LoRA)已成为大语言模型参数高效微调的标准方法,但即使是少量不安全或对抗性微调数据,也会显著削弱对齐模型的安全行为。现有安全保护型LoRA方法常依赖硬干预,如投影、剪枝、阈值处理或附加训练目标。这些方法虽能抑制不安全更新方向,但可能移除任务相关信息或需额外调参。本文提出CSULoRA,一种后处理的LoRA适配器修正方法,通过最近安全更新估计实现。该方法从对齐模型与其基础检查点间的权重偏移中估计安全对齐子空间,并将每个LoRA更新分解为完全对齐、部分对齐和离子空间成分。不同于直接丢弃子空间外成分,CSULoRA求解闭式惩罚最小变更问题,保留完全对齐成分,同时根据相对能量平滑衰减潜在不安全方向。在对抗性微调实验中,CSULoRA显著降低攻击成功率,同时保留标准LoRA微调获得的大部分性能增益。

原文摘要 · Abstract (English)

Low-rank adaptation has become a standard method for parameter-efficient fine-tuning of large language models, but even small amounts of unsafe or adversarial fine-tuning data can substantially weaken the safety behavior of aligned models. Existing safety-preserving LoRA methods often rely on hard interventions such as projection, pruning, thresholding, or additional training objectives. While these methods can suppress unsafe update directions, they may also remove task-relevant information or require extra tuning. We introduce CSULoRA, a post-hoc method for correcting trained LoRA adapters through closest safe update estimation. CSULoRA estimates a safety-aligned subspace from the weight displacement between a safety-aligned model and its corresponding base checkpoint. It then decomposes each LoRA update into fully aligned, partially aligned, and off-subspace components. Instead of discarding components outside the estimated safety subspace, CSULoRA solves a closed-form penalized minimum-change problem that preserves the fully aligned component while smoothly attenuating potentially unsafe directions according to their relative energy. In adversarial fine-tuning experiments, CSULoRA substantially reduces attack success rate while preserving most of the utility gains obtained from standard LoRA fine-tuning.

LoRA安全微调参数效率

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。