arXiv:2605.30514cs.LGcs.CL2026-05

提出首个能同时高效遗忘与保留因果知识的靶向去学习方法。

MAAT: Multi-phase Adapter-Aware Targeted Unlearning

论文配图:MAAT: Multi-phase Adapter-Aware Targeted Unlearning
图 1 · 摘自论文原文
  • 分三阶段操控LoRA适配器权重,融合梯度上升与奇异值剪枝。
  • 在5W基准上实现因果类问题遗忘率92.3%、保留率87.1%的突破性平衡。
  • 适合需精准控制模型知识删除的AI安全与可解释性研究者。

机器去学习评估存在结构性偏差:针对因果与关系知识的‘为何’类问题在CounterFact、ZSRE、TOFU、MUSE和WMDP-Cyber中占比均低于1.3%,最低仅0.06%。这种极低比例导致现有方法即使在因果知识上失效,仍可获得高平均得分,且难以察觉。本文提出5WBENCH,一个包含5,000个样本、每类5W(谁、什么、何时、何地、为何)各1,000例的均衡基准,首次使因果去学习失败可量化。基于此,我们发现现有基线无法同时实现高遗忘与高保留:激进遗忘会损害保留知识,保守方法则无法有效遗忘因果事实。‘为何’类难度源于44%的多跳推理链(其他类别≤2%)及40.1词长答案跨度带来的梯度稀释。为此,我们提出MAAT(多阶段适配器感知靶向去学习),一种基于LoRA适配器权重的三阶段框架,结合梯度投影上升、奇异值分解秩维剪枝、任务向量反向与混合KL-隐藏状态修复机制。MAAT是首个在‘为何’类因果知识上同时实现高遗忘与高保留的方法,在遗忘-保留帕累托前沿上开辟新操作点。代码已开源。

原文摘要 · Abstract (English)

Machine unlearning evaluation is structurally skewed: Why-type questions, which probe causal and relational knowledge, comprise less than 0.06% of CounterFact, 0.6% of ZSRE, and less than 1.3% of TOFU, MUSE, and WMDP-Cyber. This near-zero representation means that methods that fail on causal knowledge can score highly in aggregate, and this failure is undetectable without balanced evaluation. We present 5WBENCH, a balanced 5,000-sample benchmark with 1,000 examples per 5W category (Who, What, When, Where, Why), making causal unlearning failures quantifiable for the first time. Using 5WBENCH, we show that no existing baseline simultaneously achieves high forgetting and high retention on Why-type questions: aggressive forgetting degrades retained knowledge, while conservative methods fail to forget causal facts. Why-type difficulty stems from multi-hop reasoning chains (44% of Why entries vs. less than or equal to 2% for others) and gradient dilution over 40.1-token answer spans. We present MAAT (Multi-phase Adapter-Aware Targeted Unlearning), a three-phase framework operating on LoRA adapter weights, combining gradient-projected ascent, SVD rank-dimension pruning, task vector negation, and hybrid KL-hidden-state retain repair. MAAT is the first method to simultaneously achieve high forgetting and high retention on Why-type causal knowledge, reaching a new operating point on the forget-retain Pareto frontier. We make our code publicly available.

去学习因果推理适配器模型编辑

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。