用小模型引导大模型避开敏感信息，推理时实现无损去记忆。
Divergence Decoding: Inference-Time Unlearning via Auxiliary Models
- 用辅助模型在推理时调整大模型的输出分布，避开特定数据。
- 在多种模型和数据规模下，去记忆效果显著优于现有方法。
- 可将引导结果轻松蒸馏回原模型，适用文本与图像任务。
大型语言模型常记忆训练数据中的敏感内容，带来隐私与版权风险。现有去记忆方法或导致性能大幅下降，或对复杂查询无效。本文提出发散解码（Divergence Decoding, DD），通过小型辅助模型在推理阶段引导大模型的词元概率，使其避开特定数据。辅助模型训练简单，采用标准预训练与微调流程。实验表明，DD 在多种模型与数据规模下均显著优于当前最优基线，且可将引导结果直接蒸馏回主模型。由于该方法适用于任意概率模型，我们进一步验证其在图像生成领域的泛化能力。
原文摘要 · Abstract (English)
Large Language Models (LLMs) frequently memorize sensitive training data thereby creating significant privacy and copyright risks. Addressing these risks, i.e., removing such knowledge from an existing model checkpoint, has proven challenging as many unlearning methods lead to catastrophic utility loss or are ineffective for complex queries. We introduce Divergence Decoding (DD), a mechanism that uses small auxiliary models to steer the logits of the LLM away from specific data during inference. Training these models is straight forward, i.e., we use standard pre-training and fine-tuning setups. We find the method decisively outperforms state-of-the-art (SOTA) baselines on unlearning benchmarks across a variety of model and training dataset scales consistent with DD being an effective and inexpensive solution to unlearning. We then demonstrate that this steered distribution can be trivially distilled back into the base model. Since the method is generally applicable to any probabilistic model, we explore its efficacy outside of text generation and find evidence of generalization to the domain of images.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。