arXiv:2608.01422cs.CLcs.LG2026-08

用快速分解法实现高效模型删忆,不重训也能精准擦除特定信息。

QR-Erase: Efficient Subspace-Based Machine Unlearning with Layer Localization

论文配图:QR-Erase: Efficient Subspace-Based Machine Unlearning with Layer Localization
图 1 · 摘自论文原文
  • 用选主元QR分解定位任务相关参数,直接从模型中移除。
  • 在语音、跨语言等任务上遗忘率提升,准确率损失低于5%。
  • 适合需要快速删忆且保留通用能力的大型模型应用。

机器删忆旨在不重新训练的前提下,从已训练模型中移除特定信息。现有基于优化的方法常损害无关能力,而基于子空间的方法依赖计算成本高的奇异值分解(SVD)。我们提出QR-Erase,一种基于子空间的框架,采用选主元QR分解直接识别并移除模型参数中的任务特异性表示。进一步提出层定位版QR-Erase,仅更新包含最高任务信息密度的层。我们证明选主元QR可实现有界误差的子空间恢复,在弱谱间隙条件下,其结果逼近最优SVD解。在任务级、跨语言及语音删忆场景下,QR-Erase在遗忘-保留权衡上优于优化方法,且各项指标均在SVD的5%以内。利用低秩与层局部结构进一步提升遗忘效果(如语音遗忘集准确率从53.1%降至15.7%)。结果表明,精确子空间恢复而非最优重构即可实现有效删忆,为现代基础模型提供高效通用的SVD替代方案。

原文摘要 · Abstract (English)

Machine unlearning seeks to remove targeted information from trained models without requiring costly retraining. Existing optimization-based methods often degrade unrelated capabilities, while subspace-based approaches rely on computationally expensive singular value decompositions (SVD). We introduce QR-Erase, a subspace-based framework that uses Pivoted QR decomposition to identify and remove task-specific representations directly from model parameters. We further propose Layer-Localized QR-Erase, which restricts updates to layers containing the highest concentration of task-specific information. We show that Pivoted QR provides accurate subspace recovery with bounded error, and that under a mild spectral gap condition, the recovered subspace approaches the optimal SVD solution. Across task-level, cross-lingual, and speech unlearning, QR-Erase achieves a stronger forgetting-retention tradeoff than optimization-based methods while remaining within 5% of SVD across all metrics. Exploiting low-rank and layer-localized structure further improves forgetting (for example, reducing speech forget-set accuracy from 53.1% to 15.7%). These results demonstrate that accurate subspace recovery, rather than optimal reconstruction, is sufficient for effective unlearning and provides an efficient and general alternative to SVD-based methods for modern foundation models.

模型删忆子空间高效算法

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。