精准清除模型数学推理偏差，保留其他能力不损伤。
Mechanism-Guided Selective Unlearning for RLVR-Induced Reasoning
- 按注意力能量与梯度耦合度筛选参数，仅更新关键部分。
- 数学能力下降45→37分（显著），但GSM8K保持不变。
- 适合需要局部修正模型偏见的场景，如教育类AI优化。
我们提出MAST（机制对齐选择性靶向）方法，用于在减少副作用的前提下消除由强化学习视觉推理（RLVR）引发的推理偏差。在Qwen2.5-Math-1.5B和Qwen3-1.7B-Base的匹配微调（SFT）与RLVR检查点上，发现从SFT到RLVR的增量在词元级对数概率差上差异显著；而全参数梯度上升虽能遗忘，却会损害原有的MATH和GSM8K表现。MAST通过评估注意力投影张量的非主能量、更新幅度及遗忘梯度耦合度进行排序，仅更新前缀子集。在主模型中，MAST实现显著目标遗忘（MATH从45/150降至37/150，McNemar p=0.0078），同时保留GSM8K（+0.8 pp）和原MATH能力（-0.5 pp）。该优势在不同种子、NPO/SimNPO目标函数及Qwen3模型上均复现，而全参数遗忘会使GSM8K崩溃。
原文摘要 · Abstract (English)
We propose MAST (Mechanism-Aligned Selective Targeting), a mechanism-guided method for unlearning RLVR-induced reasoning with substantially lower collateral damage than standard full-parameter updates. In matched SFT/RLVR checkpoints on Qwen2.5-Math-1.5B and Qwen3-1.7B-Base, the SFT-to-RLVR increment differs sharply from the SFT update in token-level delta-log-probability, and full-parameter gradient ascent forgets only by damaging retain MATH and GSM8K. MAST ranks attention-projection tensors by off-principal energy, update magnitude, and forget-gradient coupling magnitude, then updates only the top-ranked subset. On the primary model, MAST induces statistically significant target forgetting (MATH forget 45/150 to 37/150; McNemar p=0.0078) while preserving GSM8K (+0.8 pp) and MATH retain (-0.5 pp). The advantage reproduces across seeds, NPO/SimNPO objectives, and Qwen3, where MAST preserves GSM8K while full-parameter unlearning collapses it.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。