提出SaLoRA，让模型微调时保持安全对齐，不破坏原有价值观。
SaLoRA: Safety-Alignment Preserved Low-Rank Adaptation
- 用安全数据预设模块，固定关键安全参数
- 在多种任务中优于现有适配器方法，保持安全对齐
- 适合需要安全微调的工业级大模型应用
随着大语言模型（LLMs）的发展和个性化需求增加，参数高效微调（PEFT）方法（如LoRA）因其降低计算成本而变得至关重要。然而，近期研究指出LoRA微调可能损害大模型的安全对齐性，给模型所有者带来重大风险。本文首先分析微调前后与安全对齐相关的特征变化机制。随后提出一种由安全数据计算得出的固定安全模块，以及针对任务的可训练参数初始化策略，称为安全对齐保持的低秩适应（SaLoRA）。与以往的LoRA及其变体不同，SaLoRA可在不破坏原始对齐的前提下实现目标模型的精准调整。实验表明，SaLoRA在多种微调任务的不同评估指标上均优于基于适配器的方法。
原文摘要 · Abstract (English)
As advancements in large language models (LLMs) continue and the demand for personalized models increases, parameter-efficient fine-tuning (PEFT) methods (e.g., LoRA) will become essential due to their efficiency in reducing computation costs. However, recent studies have raised alarming concerns that LoRA fine-tuning could potentially compromise the safety alignment in LLMs, posing significant risks for the model owner. In this paper, we first investigate the underlying mechanism by analyzing the changes in safety alignment related features before and after fine-tuning. Then, we propose a fixed safety module calculated by safety data and a task-specific initialization for trainable parameters in low-rank adaptations, termed Safety-alignment preserved Low-Rank Adaptation (SaLoRA). Unlike previous LoRA methods and their variants, SaLoRA enables targeted modifications to LLMs without disrupting their original alignments. Our experiments show that SaLoRA outperforms various adapters-based approaches across various evaluation metrics in different fine-tuning tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。