提出分层门控专家模型，高效应对未知任务流的持续学习挑战。
Hierarchically Gated Experts for Efficient Online Continual Learning
- 动态增长专家结构，避免灾难性遗忘
- 分层架构实现高效专家选择，提升推理效率
- 适合在线持续学习场景，尤其适用于数据流未知的任务
持续学习模型旨在顺序接收任务且无法访问历史数据的情况下进行学习。在线持续学习框架进一步要求任务未知，数据以单一流形式到达。本文基于已有工作，提出一种识别潜在任务的方法：门控专家（GE）算法，通过动态增长的专家集合实现新知识的获取而不会造成灾难性遗忘。进一步将GE扩展为分层门控专家（HGE），通过层级结构组织专家，可高效为每个数据样本选择最优专家。在标准持续学习基准测试中，GE与HGE表现与现有方法相当，其中HGE更具效率。
原文摘要 · Abstract (English)
Continual Learning models aim to learn a set of tasks under the constraint that the tasks arrive sequentially with no way to access data from previous tasks. The Online Continual Learning framework poses a further challenge where the tasks are unknown and instead the data arrives as a single stream. Building on existing work, we propose a method for identifying these underlying tasks: the Gated Experts (GE) algorithm, where a dynamically growing set of experts allows for new knowledge to be acquired without catastrophic forgetting. Furthermore, we extend GE to Hierarchically Gated Experts (HGE), a method which is able to efficiently select the best expert for each data sample by organising the experts into a hierarchical structure. On standard Continual Learning benchmarks, GE and HGE are able to achieve results comparable with current methods, with HGE doing so more efficiently.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。