用影响驱动重加权，高效清除大模型特定数据记忆。
RapidUn: Influence-Driven Parameter Reweighting for Efficient Large Language Model Unlearning
- 基于样本影响快速估算，生成可调节参数更新权重
- 比完整重训练快100倍，在多个数据集上表现更优
- 适合需要高效、稳定遗忘的工业级大模型应用
从大语言模型中移除特定数据的影响仍具挑战性，因完整重训练成本高，而现有近似遗忘方法常不稳定。当需遗忘的数据集较小或分布不均时，问题更严重。我们提出RapidUn，一种基于影响驱动且参数高效的遗忘框架。首先通过快速估计模块估算每个样本的影响，再将这些得分映射为自适应更新权重，指导选择性参数更新——既能消除有害行为，又能保留通用知识。在Mistral-7B和Llama-3-8B模型上，针对Dolly-15k和Alpaca-57k数据集，RapidUn效率比全量重训练最高提升100倍，并在分布内与分布外遗忘任务中持续优于Fisher、GA和LoReUn。这些结果确立了影响引导的参数重加权作为可扩展、可解释的大模型遗忘新范式。
原文摘要 · Abstract (English)
Removing specific data influence from large language models (LLMs) remains challenging, as retraining is costly and existing approximate unlearning methods are often unstable. The challenge is exacerbated when the forget set is small or imbalanced. We introduce RapidUn, an influence-driven and parameter-efficient unlearning framework. It first estimates per-sample influence through a fast estimation module, then maps these scores into adaptive update weights that guide selective parameter updates -- forgetting harmful behavior while retaining general knowledge. On Mistral-7B and Llama-3-8B across Dolly-15k and Alpaca-57k, RapidUn achieves up to 100 times higher efficiency than full retraining and consistently outperforms Fisher, GA, and LoReUn on both in-distribution and out-of-distribution forgetting. These results establish influence-guided parameter reweighting as a scalable and interpretable paradigm for LLM unlearning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。