无需微调模型，用规则实时清除特定数据影响。
ICCU: In-Context Continual Unlearning via Pattern-Induced Refusal Rules
- 从待删除数据中提取可读规则，推理时通过提示或过滤应用。
- 支持连续请求，知识抑制效果显著且不损失模型性能。
- 适合需要持续删除敏感数据的生产环境使用。
机器遗忘旨在移除训练语言模型中特定数据的影响。在实际部署中，遗忘请求常以序列形式到达，传统基于微调的方法存在成本高、性能退化及请求间干扰等问题。为此，我们提出ICCU（上下文持续遗忘），一种基于上下文的持续遗忘框架：从遗忘数据中提取可读的拒绝规则，并在推理时作为过滤器或系统提示应用，无需修改模型参数。由于规则以无序并集方式累积，ICCU具备组合性且无跨请求干扰，原始遗忘数据可在规则生成后丢弃。大量实验表明，ICCU能有效抑制目标知识，同时保持模型效用，可扩展至连续请求，且对改写和跨语言查询具有鲁棒性。
原文摘要 · Abstract (English)
Machine unlearning aims to remove the influence of specific data from trained language models. In real-world deployments, unlearning requests often arrive sequentially, which challenges existing fine-tuning-based methods: fine-tuning each request is costly, accumulates utility loss, and may cause cross-request interference. To address these issues, we propose ICCU (In-Context Continual Unlearning), an in-context continual unlearning framework that induces readable refusal rules from unlearning datasets and applies them at inference time either as a filter or via the system prompt, without modifying model parameters. Because rules are accumulated as an order-independent union, ICCU is compositional and free of cross-request interference, and the original forget-set data can be discarded after rule induction. Extensive experiments show that ICCU effectively suppresses target knowledge while preserving utility, scales across sequential requests, and remains robust to paraphrased and cross-lingual queries.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。