通过对齐模型内部表示，实现无数据持续学习，防止灾难性遗忘。
RECALL: REpresentation-aligned Catastrophic-forgetting ALLeviation via Hierarchical Model Merging
- 基于分层隐藏表示计算模型相似性，自适应融合参数
- 在5个NLP任务上显著提升知识保留与泛化能力
- 无需历史数据和任务标签，适合大规模模型演进
我们发现大型语言模型（LLMs）的内部表示可作为已学知识的可靠代理，提出RECALL——一种无需历史数据的表示感知模型融合框架，用于持续学习。该方法基于聚类典型样本的分层隐藏表示，计算模型间相似性，并执行自适应、分层参数融合，以对齐各模型知识。该设计使浅层保持领域通用特征，深层支持任务特定适应。与以往需任务标签或存在性能折衷的方法不同，RECALL实现无缝多领域整合并强抵抗灾难性遗忘。在五个NLP任务及多个持续学习场景中的大量实验表明，其在知识保留与泛化能力上均优于基线，为演进式LLM提供可扩展且无数据的解决方案。
原文摘要 · Abstract (English)
We unveil that internal representations in large language models (LLMs) serve as reliable proxies of learned knowledge, and propose RECALL, a novel representation-aware model merging framework for continual learning without access to historical data. RECALL computes inter-model similarity from layer-wise hidden representations over clustered typical samples, and performs adaptive, hierarchical parameter fusion to align knowledge across models. This design enables the preservation of domain-general features in shallow layers while allowing task-specific adaptation in deeper layers. Unlike prior methods that require task labels or incur performance trade-offs, RECALL achieves seamless multi-domain integration and strong resistance to catastrophic forgetting. Extensive experiments across five NLP tasks and multiple continual learning scenarios show that RECALL outperforms baselines in both knowledge retention and generalization, providing a scalable and data-free solution for evolving LLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。