arXiv:2609.05075cs.AI2026-09

让预训练模型在持续学习中既灵活又稳定,提升长期适应能力。

MePo++: Unifying Representation Refinement and Reconciliation for General Continual Learning

论文配图:MePo++: Unifying Representation Refinement and Reconciliation for General Continual Learning
图 1 · 摘自论文原文
  • 通过无监督元优化构建伪持续序列,增强表征可塑性。
  • 在线特征与预训练几何对齐,缓解模糊数据流中的表征漂移。
  • 适用于多种预训练模型和数据集,通用性强,适合真实场景持续学习。

通用持续学习(GCL)旨在不依赖任务标识、明确边界或重复访问历史数据的情况下,从不断演化的数据流中学习,是实现持续智能的真实而具有挑战性的设置。尽管预训练模型(PTMs)为应对监督信息有限和非平稳特性提供了丰富的先验知识,但现有基于PTM的方法通常直接适配预训练表征,忽略了两个关键差距:上游预训练与下游持续适应之间的表征错位,以及在模糊数据流下传统输出对齐的不可靠性。为此,我们提出MePo++,一个统一的后训练框架,通过表征精炼与调和,弥合预训练知识与下游GCL之间的鸿沟。MePo++引入两个互补组件:MetaPrep,通过在伪持续序列上进行无监督元精炼,提升表征对持续适应的可塑性;StreamAlign,通过将动态演变的在线特征与稳定的预训练几何结构对齐,强化表征稳定性。通过在适应前提升表征可学习性,并在持续学习过程中保持对齐,MePo++使PTM在学习新概念时具备灵活性,同时在演化数据流中保持稳定性。在多种预训练模型、数据集和持续学习基线上的实验表明,MePo++在基于PTM的GCL中表现出一致的有效性与普适性。代码已开源:https://github.com/SunGL001/MePo_Plus。

原文摘要 · Abstract (English)

General continual learning (GCL) aims to learn from evolving data streams without task identities, explicit boundaries, or repeated access to previous data, making it a realistic yet challenging setting for continual intelligence. Although pretrained models (PTMs) provide rich prior knowledge for addressing the limited supervision and non-stationary nature of GCL, existing PTM-based methods often directly adapt pretrained representations and overlook two critical gaps: the misalignment between upstream pretraining and downstream continual adaptation, and the unreliability of conventional output alignment under blurry streams. Here we propose MePo++, a unified post-training framework that bridges pretrained knowledge and downstream GCL through representation refinement and reconciliation. MePo++ introduces two complementary components: MetaPrep, which improves representation plasticity for continual adaptation through unsupervised meta-refinement over pseudo continual sequences; and StreamAlign, which reinforces representation stability by reconciling evolving online features with a stable pretrained geometry. By improving representation learnability before adaptation and preserving alignment during continual learning, MePo++ enables PTMs to remain both plastic for new concepts and stable over evolving streams. Experiments across diverse PTMs, datasets, and continual learning baselines demonstrate the consistent effectiveness and generality of MePo++ for PTM-based GCL. Our code is available at https://github.com/SunGL001/MePo_Plus.

持续学习预训练模型表征优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。