参数高效微调可能削弱大模型的安全与公平性,需警惕。
Efficiency vs. Alignment: Investigating Safety and Fairness Risks in Parameter-Efficient Fine-Tuning of LLMs
- 系统测试四种方法在11类安全风险和9维公平性上的影响
- 提示类方法更易降低安全性和公平性,适配器类较稳定
- 选基模型和方法需谨慎,安全不等于公平,要逐项审计
组织越来越多地从HuggingFace等公共仓库中适配大语言模型(LLMs)以完成下游任务。已有研究显示,即使在良性数据集上微调也会削弱安全对齐性,引发一个实际问题:良性参数高效微调(PEFT)是否也影响安全与公平?我们首次进行了大规模、系统的研究,表明良性PEFT确实会显著改变这两方面。我们在四个指令微调模型家族(Meta-Llama-3-8B、Qwen2.5-7B、Mistral-7B、Gemma-7B)上,使用四种常用PEFT方法(LoRA、IA3、Prompt-Tuning、P-Tuning),共评估了235个对话微调变体,涵盖十一类安全风险和九种公平性维度。为评估泛化能力,还引入一个紧凑的编码任务扩展,包含96个额外微调模型。结果表明,良性PEFT可引发有害对齐偏移。基于适配器的方法(如LoRA、IA3)通常更安全且对公平性影响较小,而基于提示的方法更常降低安全性并恶化公平性表现。基模型选择强烈调节这些效应:LLaMA相对稳定,Qwen略有提升,Gemma安全下降最剧烈,Mistral最不稳定。编码任务扩展同样产生对齐偏移,但与对话任务相比,任务层级差异有限。总体而言,安全改进不能可靠传递到公平性,且无单一配置能优化所有公平性指标。研究支持实用指南:善意意图不保证安全行为;部署时应选用已良好对齐的基模型,优先适配器类PEFT,并在类别层面审计安全与公平性。
原文摘要 · Abstract (English)
Organizations increasingly adapt Large Language Models (LLMs) from public repositories such as HuggingFace to downstream tasks. Prior work shows that even fine-tuning on benign datasets can weaken safety alignment, raising a practical question: does benign parameter-efficient fine-tuning (PEFT) also affect safety and fairness? We present the first large-scale, systematic study showing that benign PEFT can significantly alter both. We fine-tune four instruction-tuned model families (Meta-Llama-3-8B, Qwen2.5-7B, Mistral-7B, and Gemma-7B) with four widely used PEFT methods: LoRA, IA3, Prompt-Tuning, and P-Tuning. In total, we evaluate 235 conversationally fine-tuned variants across eleven safety hazard categories and nine fairness dimensions. We assess generalization beyond conversational tuning by incorporating a compact extension focused on coding tasks, involving 96 additional fine-tuned models. Results show that benign PEFT can induce detrimental alignment shifts. Adapter-based methods (LoRA, IA3) are generally safer and less disruptive to fairness, whereas prompt-based methods more often reduce safety and worsen fairness accuracy. Base model choice strongly moderates these effects: LLaMA is comparatively stable, Qwen shows modest gains, Gemma exhibits the steepest safety decline, and Mistral is the most variable. The coding-task extension also produces alignment shifts relative to base models, but matched comparisons with the conversational task reveal limited task-level differences. Overall, safety improvements do not reliably transfer to fairness, and no single configuration optimizes every fairness metric. These findings support a practical guideline for safety-critical deployment: benign intent does not guarantee safe behaviour; start from a well-aligned base model, favour adapter-based PEFT, and audit safety and fairness at the category level.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。