arXiv:2502.11609cs.LG2025-02NeurIPS被引 2

通过任务间可迁移性嵌入,提升持续学习中的正向与反向迁移能力。

Exploiting Task Relationships in Continual Learning via Transferability-Aware Task Embeddings

  • 基于信息论构建可在线计算的任务嵌入H-embedding,捕捉任务间关系。
  • 在CIFAR-100、ImageNet-R和DomainNet上优于主流方法,显著提升迁移性能。
  • 仅需存储低维任务嵌入,支持高效端到端训练,适合实际部署。

持续学习(CL)是当前深度神经网络应用中的关键课题,理想的CL性能需要同时实现较强的正向与反向迁移。现有方法多聚焦于任务模型本身,如通过正则化更新或分离特定任务与共享组件,但往往忽视利用任务间关系来增强迁移。为此,我们提出一种可迁移性感知的任务嵌入——H-embedding,并在此基础上构建超网络框架,为持续学习任务学习任务条件化的模型权重。H-embedding源自信息论的可迁移性度量,具备在线计算与低复杂度特性。本方法具有显著实用性,每个任务仅需存储低维嵌入,且支持高效端到端训练。在CIFAR-100、ImageNet-R和DomainNet等基准上的广泛评估表明,该框架显著优于多种基线与最先进方法,展现出捕捉并利用内在任务关系的强大潜力。代码已公开于https://github.com/viki760/Hembedding_Guided_Hypernet。

原文摘要 · Abstract (English)

Continual learning (CL) has been a critical topic in contemporary deep neural network applications, where higher levels of both forward and backward transfer are desirable for an effective CL performance. Existing CL strategies primarily focus on task models, either by regularizing model updates or by separating task-specific and shared components, while often overlooking the potential of leveraging inter-task relationships to enhance transfer. To address this gap, we propose a transferability-aware task embedding, termed H-embedding, and construct a hypernet framework under its guidance to learn task-conditioned model weights for CL tasks. Specifically, H-embedding is derived from an information theoretic measure of transferability and is designed to be online and easy to compute. Our method is also characterized by notable practicality, requiring only the storage of a low-dimensional task embedding per task and supporting efficient end-to-end training. Extensive evaluations on benchmarks including CIFAR-100, ImageNet-R, and DomainNet show that our framework performs prominently compared to various baseline and SOTA approaches, demonstrating strong potential in capturing and utilizing intrinsic task relationships. Our code is publicly available at https://github.com/viki760/Hembedding_Guided_Hypernet.

持续学习任务嵌入可迁移性超网络

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。