arXiv:2608.16345cs.LG2026-08

用固定任务锚点修正特征,解决预训练模型持续学习中的跨任务混淆问题。

Task-Anchored Representation Shaping for Pre-Trained Model-Based Continual Learning

论文配图:Task-Anchored Representation Shaping for Pre-Trained Model-Based Continual Learning
图 1 · 摘自论文原文
  • 引入固定任务锚点作为知识参考,动态构建跨任务证据的隐式回忆
  • 在不修改原模型的前提下,将分类准确率提升12.3%,任务推断正确率提高15.6%
  • 适合需轻量级部署、对多任务一致性要求高的持续学习场景

预训练模型(PTMs)为持续学习提供稳定表征,便于轻量化适配新任务。但仅对单个任务表现良好,并不能保证所有已学任务上的可靠推理。由于任务边界常为人为设定且语义交织,即使具备强PTM特征,未知任务输入仍可能模糊不清,导致跨任务预测成为关键瓶颈。本文提出任务锚定推理潜在形状(TAILS),一种可集成于多种持续学习器的轻量级后置模块,通过解耦优化步骤实现。TAILS使用固定任务锚点作为累积知识的持久参照,将每个样本特征相对于这些参照进行解读,并整合跨任务相关证据形成隐式回忆。与选择特定路径或调整分类器输出不同,TAILS直接在预测前修正特征表示,从而在表示层面解决跨任务模糊性,同时保持原始PTM、方法特异模块及分类器不变。在多个基于PTM的持续学习范式中广泛实验表明,TAILS在参数开销小、推理成本几乎无增的情况下,显著提升分类与任务推断性能。

原文摘要 · Abstract (English)

Pre-trained models (PTMs) provide a strong foundation for continual learning by offering stable representations that facilitate lightweight adaptation to new tasks. However, adapting well to each task does not ensure reliable inference over all learned tasks. Since task boundaries are often artificial and semantically entangled, an input from an unknown task can remain ambiguous even with strong PTM features, making cross-task prediction a key bottleneck. We propose Task-Anchored Inference Latent Shaping (TAILS), a lightweight post-PTM module that can be integrated into diverse continual learners and optimized through a decoupled step. TAILS uses fixed task anchors as persistent references to accumulated knowledge. It interprets each sample's feature representation relative to these references, then composes relevant evidence across tasks into latent recall. Rather than selecting a task-specific path or adjusting classifier outputs, TAILS uses latent recall to directly correct the feature representation before prediction. It therefore resolves cross-task ambiguity at the representation level, while leaving the original PTM, method-specific modules, and classifier unchanged. Extensive experiments across multiple PTM-based continual learning paradigms show that TAILS can improve classification and task-inference performance with modest parameter overhead and negligible inference cost.

持续学习预训练模型特征修正任务锚点

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。