arXiv:2411.11932cs.LGcs.AI2024-11ACL被引 3

发现大模型'假遗忘'现象,通过引导推理恢复旧任务能力。

Unveiling and Addressing Pseudo Forgetting in Large Language Models

  • 识别出性能下降源于指令失效而非能力丧失
  • 用外部推理或无意义后缀可恢复旧任务表现
  • 提出动态重放框架,适合持续学习场景

尽管已有大量研究致力于缓解持续学习中的灾难性遗忘,但其内在机制仍不明确。本文揭示了‘伪遗忘’现象:旧任务性能下降并非因能力丢失,而是指令无法激活相应模型能力。我们通过两种简单干预证明性能可恢复:(1)提供部分外部正确推理过程;(2)在原始指令后添加语义无关的后缀,以引导生成正确推理。通过对内部推理生成机制的实证分析,发现出现伪遗忘的模型在推理生成中表现出较弱的指令依赖性,导致内在能力未能有效激活。基于此,我们提出基于推理引导难度的重放框架(RGD-R),根据模型利用内在能力的能力动态分配重放数据。实验表明,RGD-R能有效缓解伪遗忘,同时保持模型的可塑性。

原文摘要 · Abstract (English)

Although substantial efforts have been made to mitigate catastrophic forgetting in continual learning, the intrinsic mechanisms are not well understood. In this work, we demonstrate the existence of "pseudo forgetting": the performance degradation on previous tasks is not attributed to a loss of capabilities, but rather to the failure of the instructions to activate the appropriate model abilities. We show that the model's performance on previous tasks can be restored through two simple interventions: (1) providing partial external correct rationale, and (2) appending semantically meaningless suffixes to the original instructions, to guide the generation of correct rationales. Through empirical analysis of the internal mechanisms governing rationale generation, we reveal that models exhibiting pseudo forgetting show reduced instruction dependence during rationale generation, leading to suboptimal activation of their inherent capabilities. Based on this insight, we propose Rationale-Guidance Difficulty based Replay (RGD-R) framework that dynamically allocates replay data based on the model's ability to correctly leverage the intrinsic capabilities. Experimental results demonstrate that RGD-R effectively mitigates pseudo forgetting while maintaining model plasticity.

持续学习大模型推理引导

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。