用贝叶斯方法动态判断数据安全,自适应防御大模型有害微调。
Adaptive Defense against Harmful Fine-Tuning for Large Language Models via Bayesian Data Scheduler

- 将安全防御建模为贝叶斯推断,实时评估每条数据的风险
- 无需预设攻击模型,在多种攻击下均达顶尖防御效果
- 适合需灵活应对未知攻击的模型服务场景
有害微调对大模型微调即服务带来重大安全风险。现有防御策略依赖攻击模拟构建鲁棒性,但存在两大根本缺陷:(i) 攻击模拟难以扩展至未预见的攻击类型,因无法提前预测未知攻击;(ii) 对不同攻击设置适应性差,因模拟无法捕捉其多样性和复杂性。为此,我们提出贝叶斯数据调度器(BDS),一种无需攻击模拟的自适应微调阶段防御策略。BDS 将有害微调防御建模为贝叶斯推断问题,学习每个数据点安全属性的后验分布,条件于微调和对齐数据集。微调过程通过采样后验得到的安全权重对数据加权,从而削弱有害数据影响。利用贝叶斯推断的后验特性,后验条件于微调数据集,使 BDS 能根据具体数据集定制防御,实现自适应。此外,我们引入基于近似贝叶斯学习的神经调度器,实现新数据的高效迁移而无需重训。在多种攻击与防御设置下的全面实验表明,该方法性能达到当前最优。代码已开源:https://github.com/Egg-Hu/Bayesian-Data-Scheduler。
原文摘要 · Abstract (English)
Harmful fine-tuning poses critical safety risks to fine-tuning-as-a-service for large language models. Existing defense strategies preemptively build robustness via attack simulation but suffer from fundamental limitations: (i) the infeasibility of extending attack simulations beyond bounded threat models due to the inherent difficulty of anticipating unknown attacks, and (ii) limited adaptability to varying attack settings, as simulation fails to capture their variability and complexity. To address these challenges, we propose Bayesian Data Scheduler (BDS), an adaptive tuning-stage defense strategy with no need for attack simulation. BDS formulates harmful fine-tuning defense as a Bayesian inference problem, learning the posterior distribution of each data point's safety attribute, conditioned on the fine-tuning and alignment datasets. The fine-tuning process is then constrained by weighting data with their safety attributes sampled from the posterior, thus mitigating the influence of harmful data. By leveraging the post hoc nature of Bayesian inference, the posterior is conditioned on the fine-tuning dataset, enabling BDS to tailor its defense to the specific dataset, thereby achieving adaptive defense. Furthermore, we introduce a neural scheduler based on amortized Bayesian learning, enabling efficient transfer to new data without retraining. Comprehensive results across diverse attack and defense settings demonstrate the state-of-the-art performance of our approach. Code is available at https://github.com/Egg-Hu/Bayesian-Data-Scheduler.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。