提出T3-Unlearning方法，让生成模型更精准地删除特定数据。
Temper-Then-Tilt: Principled Unlearning for Generative Models through Tempering and Classifier Guidance
- 先降温再引导，通过两步推理修正生成分布
- 在TOFU基准上遗忘质量提升，仅训练少量参数
- 适合需要精准删除数据的生成模型应用
我们研究大模型中的机器遗忘问题，将任务定义为对目标分布的密度比估计，而非监督微调。尽管分类器引导是近似该比值的标准方法，但在有限样本下，当遗忘集对应尖锐集中数据分布时可能失效。为此，我们提出温控-倾斜遗忘（T3-Unlearning），冻结基础模型，采用两步推断：(i) 对基础分布进行温度调节以压平高置信度尖峰；(ii) 使用轻量分类器区分保留与遗忘样本，对调节后的分布进行倾斜。理论分析给出了有限样本下代理分类器风险与遗忘误差的关联性保证，证明温度调节对集中分布遗忘是必要条件。在TOFU基准上的实验表明，T3-Unlearning在遗忘质量和生成效用上优于现有基线，同时仅训练极小部分参数且运行时间极低。
原文摘要 · Abstract (English)
We study machine unlearning in large generative models by framing the task as density ratio estimation to a target distribution rather than supervised fine-tuning. While classifier guidance is a standard approach for approximating this ratio and can succeed in general, we show it can fail to faithfully unlearn with finite samples when the forget set represents a sharp, concentrated data distribution. To address this, we introduce Temper-Then-Tilt Unlearning (T3-Unlearning), which freezes the base model and applies a two-step inference procedure: (i) tempering the base distribution to flatten high-confidence spikes, and (ii) tilting the tempered distribution using a lightweight classifier trained to distinguish retain from forget samples. Our theoretical analysis provides finite-sample guarantees linking the surrogate classifier's risk to unlearning error, proving that tempering is necessary to successfully unlearn for concentrated distributions. Empirical evaluations on the TOFU benchmark show that T3-Unlearning improves forget quality and generative utility over existing baselines, while training only a fraction of the parameters with a minimal runtime.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。