arXiv:2511.09880cs.CLcs.CR2025-11中稿 · IEEE Symposium on …

让微调后的大模型保持安全,无需重新训练。

EnchTable: Unified Safety Alignment Transfer in Fine-tuned Large Language Models

  • 用神经正切核方法提取安全向量,分离安全与任务能力。
  • 在11个数据集上测试,安全率显著优于厂商原生模型。
  • 适合需要高安全性的代码、医疗、数学等场景使用。

许多机器学习模型通过微调大语言模型(LLMs)在代码生成、生物医学分析和数学问题求解等专业领域达到高性能,但该过程常导致安全对齐系统性退化,增加有害输出风险。为此,我们提出EnchTable框架,可在不需大量重训的前提下,迁移并维持下游LLM的安全对齐。EnchTable采用基于神经正切核(NTK)的安全向量蒸馏方法,将安全约束与任务推理解耦,确保在不同模型架构和规模下的兼容性。此外,其干扰感知融合技术有效平衡安全与实用性,最大限度减少性能损失。我们在三个任务领域和三种不同LLM架构上实现完整原型,并在11个多样化数据集上进行广泛实验,评估模型的实用性与安全性。测试涵盖多个厂商的LLM,验证了EnchTable的泛化能力。同时,该框架对静态和动态越狱攻击具有强抵抗力,在缓解对抗性提示方面优于厂商发布的安全模型。与六种参数修改方法及两种推理时对齐基线相比,EnchTable在降低不安全率、提升实用分方面表现更优,且在不同任务域中具有普适性。此外,可无缝集成至各类部署流程,开销极小。

原文摘要 · Abstract (English)

Many machine learning models are fine-tuned from large language models (LLMs) to achieve high performance in specialized domains like code generation, biomedical analysis, and mathematical problem solving. However, this fine-tuning process often introduces a critical vulnerability: the systematic degradation of safety alignment, undermining ethical guidelines and increasing the risk of harmful outputs. Addressing this challenge, we introduce EnchTable, a novel framework designed to transfer and maintain safety alignment in downstream LLMs without requiring extensive retraining. EnchTable leverages a Neural Tangent Kernel (NTK)-based safety vector distillation method to decouple safety constraints from task-specific reasoning, ensuring compatibility across diverse model architectures and sizes. Additionally, our interference-aware merging technique effectively balances safety and utility, minimizing performance compromises across various task domains. We implemented a fully functional prototype of EnchTable on three different task domains and three distinct LLM architectures, and evaluated its performance through extensive experiments on eleven diverse datasets, assessing both utility and model safety. Our evaluations include LLMs from different vendors, demonstrating EnchTable's generalization capability. Furthermore, EnchTable exhibits robust resistance to static and dynamic jailbreaking attacks, outperforming vendor-released safety models in mitigating adversarial prompts. Comparative analyses with six parameter modification methods and two inference-time alignment baselines reveal that EnchTable achieves a significantly lower unsafe rate, higher utility score, and universal applicability across different task domains. Additionally, we validate EnchTable can be seamlessly integrated into various deployment pipelines without significant overhead.

大模型安全微调对齐

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。