用户可用自然语言指令让大模型即时删除特定知识，无需重训练。
RePAIR: Interactive Machine Unlearning through Prompt-Aware Model Repair
- 通过自然语言指令触发模型在推理时自我修复，实现用户主导的遗忘。
- 在有害信息消除任务中遗忘准确率接近0，同时保留模型性能达84.47%。
- 无需训练、单样本即可完成更新，适合手机等设备本地执行。
大语言模型在大规模网络语料预训练中会吸收有害知识、错误信息和个人数据，且缺乏选择性清除机制。现有机器遗忘方法依赖服务提供商重训练、专用保留数据集和直接干预，使终端用户无法自主控制自身数据。本文提出交互式机器遗忘（IMU）新范式，让用户在推理阶段通过自然语言指令让模型遗忘特定知识。为此构建了RePAIR框架，包含：（i）检测遗忘意图的看守模型，（ii）生成修复操作的外科医生模型，（iii）自主更新参数的患者模型。核心是无训练、单样本的STAMP方法，通过伪逆闭式更新将MLP激活向拒绝子空间引导。其低秩变体将计算复杂度从O(d^3)降至O(r^3 + r^2 * d)，实现约3倍于训练基线的加速。在有害知识抑制、错误信息修正与个人数据擦除任务中，RePAIR达成近乎零遗忘得分（Acc_f = 0.00, F-RL = 0.00），同时保留模型效用（Acc_r最高84.47，R-RL最高0.88），优于六种先进基线。结果证明RePAIR是高效实用的用户驱动模型编辑框架，推动可解释、本地化的知识控制，未来可拓展至多模态基础模型。
原文摘要 · Abstract (English)
Large language models (LLMs) inherently absorb harmful knowledge, misinformation, and personal data during pretraining on large-scale web corpora, with no native mechanism for selective removal. While machine unlearning offers a principled solution, existing approaches are provider-centric, requiring retraining pipelines, curated retain datasets, and direct intervention by model service providers (MSPs), thereby excluding end users from controlling their own data. We introduce Interactive Machine Unlearning (IMU), a new paradigm in which users can instruct LLMs to forget targeted knowledge through natural language at inference time. To realize IMU, we propose RePAIR, a prompt-aware model repair framework comprising (i) a watchdog model for unlearning intent detection, (ii) a surgeon model for generating repair procedures, and (iii) a patient model whose parameters are updated autonomously. At the core of RePAIR, we develop Steering Through Activation Manipulation with PseudoInverse (STAMP), a training-free, single-sample unlearning method that redirects MLP activations toward a refusal subspace via closed-form pseudoinverse updates. Its low-rank variant reduces computational complexity from O(d^3) to O(r^3 + r^2 * d), enabling efficient on-device unlearning with up to ~3x speedup over training-based baselines. Extensive experiments across harmful knowledge suppression, misinformation correction, and personal data erasure demonstrate that RePAIR achieves near-zero forget scores (Acc_f = 0.00, F-RL = 0.00) while preserving model utility (Acc_r up to 84.47, R-RL up to 0.88), outperforming six state-of-the-art baselines. These results establish RePAIR as an effective and practical framework for user-driven model editing, advancing transparent and on-device control over learned knowledge, with potential extensions to multimodal foundation models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。