arXiv:2510.00761cs.LG2025-10被引 3

简化优化器反而让大模型遗忘更可靠,抗干扰能力更强。

Downgrade to Upgrade: Optimizer Simplification Enhances Robustness in LLM Unlearning

  • 用零阶或压缩梯度优化器,降低信息利用程度,增强遗忘稳定性。
  • 在MUSE和WMDP数据集上,新方法使遗忘效果抗量化、微调等扰动。
  • 适合关注隐私保护与模型安全的从业者,尤其在部署前需防篡改场景。

大语言模型(LLM)遗忘旨在精准移除模型中的不良数据影响,同时保留其在无关任务上的性能。然而,现有方法存在脆弱性:权重量化或微调等后续操作常导致遗忘失效。本文从优化器角度切入,发现优化器的“等级”(零阶至二阶)直接影响遗忘鲁棒性。出人意料的是,降低优化器等级(如使用梯度符号或零阶方法)虽带来更嘈杂的更新,却能引导模型收敛到更难被扰动的损失景观区域,从而提升鲁棒性。通过关联零阶方法与随机平滑,进一步揭示其天然优势。据此提出混合优化器,融合一阶与零阶更新,在保持遗忘效果的同时显著增强鲁棒性。在MUSE和WMDP基准上,多种遗忘算法验证该方法可实现更稳定的遗忘,且不牺牲质量。

原文摘要 · Abstract (English)

Large language model (LLM) unlearning aims to surgically remove the influence of undesired data or knowledge from an existing model while preserving its utility on unrelated tasks. This paradigm has shown promise in addressing privacy and safety concerns. However, recent findings reveal that unlearning effects are often fragile: post-unlearning manipulations such as weight quantization or fine-tuning can quickly neutralize the intended forgetting. Prior efforts to improve robustness primarily reformulate unlearning objectives by explicitly assuming the role of vulnerability sources. In this work, we take a different perspective by investigating the role of the optimizer, independent of unlearning objectives and formulations, in shaping unlearning robustness. We show that the 'grade' of the optimizer, defined by the level of information it exploits, ranging from zeroth-order (gradient-free) to first-order (gradient-based) to second-order (Hessian-based), is tightly linked to the resilience of unlearning. Surprisingly, we find that downgrading the optimizer, such as using zeroth-order methods or compressed-gradient variants (e.g., gradient sign-based optimizers), often leads to stronger robustness. While these optimizers produce noisier and less precise updates, they encourage convergence to harder-to-disturb basins in the loss landscape, thereby resisting post-training perturbations. By connecting zeroth-order methods with randomized smoothing, we further highlight their natural advantage for robust unlearning. Motivated by these insights, we propose a hybrid optimizer that combines first-order and zeroth-order updates, preserving unlearning efficacy while enhancing robustness. Extensive experiments on the MUSE and WMDP benchmarks, across multiple LLM unlearning algorithms, validate that our approach achieves more resilient forgetting without sacrificing unlearning quality.

大模型遗忘优化器设计鲁棒性隐私安全

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。