arXiv:2605.20296cs.LGcs.AI2026-05被引 1

修复微调导致的能力损失,无需重新训练。

Spectral Unforgetting: Post-Hoc Recovery of Damaged Capabilities Without Retraining

论文配图:Spectral Unforgetting: Post-Hoc Recovery of Damaged Capabilities Without Retraining
图 1 · 摘自论文原文
  • 通过谱分析保留微调中的有效更新,滤除无用噪声。
  • 在14个任务中实现最佳综合修复效果,保持目标任务性能。
  • 适合需要修复模型退化但无法重训的场景。

微调语言模型常导致未显式涉及的能力下降,称为灾难性遗忘。本文提出仅使用预训练检查点 $W_{\mathrm{base}}$ 及其微调版本 $W_{\mathrm{ft}}$ 的后处理修复方法 DG-Hard。该方法将微调更新 $Δ = W_{\mathrm{ft}} - W_{\mathrm{base}}$ 视为嵌入在独立同分布噪声中的低秩任务信号,利用 Donoho-Gavish 硬阈值对每个权重差矩阵的奇异值进行过滤,保留高能量结构部分,移除谱主干。修复过程为闭式SVD滤波,无需数据依赖调参。评估上引入分块条件指标,分别衡量恢复、保持、非损伤和目标任务保留。在14个(模型, 任务)组合及九个跨域基准上,DG-Hard 在后处理基线中表现最优。此外,即使无对齐数据,也能恢复微调造成的三个安全轴上的对齐退化。结果表明,部分能力损失并非特化的必然代价,而是可移除的权重更新谱残余。

原文摘要 · Abstract (English)

Fine-tuning a language model for a target task routinely degrades capabilities the training data never explicitly threatened. We study this phenomenon, known as catastrophic forgetting, and propose a post-hoc repair solution that uses only the pretrained checkpoint $W_{\mathrm{base}}$ and its fine-tuned descendant $W_{\mathrm{ft}}$. The goal is not merely to revert the model toward the base checkpoint, but to recover capabilities damaged by fine-tuning while preserving both the target-task gains and any beneficial held-out improvements. We introduce DG-Hard, a checkpoint-only spectral repair method for the fine-tuning update $Δ= W_{\mathrm{ft}} - W_{\mathrm{base}}$. DG-Hard treats $Δ$ as a low-rank task-aligned signal embedded in an IID-like noise residual that gradient descent has no incentive to remove, and applies the Donoho-Gavish hard singular-value threshold to each weight-delta matrix, keeping the structured high-energy part of the update and removing the spectral bulk. This reduces repair to a closed-form SVD filtering step requiring no data-dependent tuning. A central difficulty is evaluation: average accuracy hides per-benchmark failures, while naive recovery scores reward models that simply revert toward the base. We therefore introduce a partition-conditional metric that separately tracks healing, preservation, non-damage, and target-task retention. Across $14$ (model, task) settings and nine cross-domain held-out benchmarks, DG-Hard achieves the strongest balanced repair among post-hoc baselines. DG-Hard also restores safety alignment degraded by benign fine-tuning on three independent safety axes, despite using no alignment data. These results suggest that part of fine-tuning-induced capability loss is not an unavoidable consequence of specialization, but a removable spectral residue in the weight update itself.

模型修复灾难性遗忘谱分析后处理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。