TAME通过任务感知选择专家，实现持续学习中适应与记忆的平衡。
Task-Aware Multi-Expert Architecture For Lifelong Deep Learning
- 根据任务相似性动态选择最优专家，提升适应能力
- 在CIFAR-100上新任务准确率提升，旧任务性能几乎不下降
- 适合长期迭代训练、需保留历史知识的场景
持续深度学习（Lifelong Deep Learning, LDL）旨在让神经网络在顺序学习多个任务的同时保留已有知识。本文提出任务感知多专家架构（Task-Aware Multi-Expert, TAME），利用任务相似性指导专家选择与知识迁移。TAME维护一组预训练神经网络，为每个新任务激活最相关的专家；共享全连接层融合所选专家特征以生成预测。为缓解灾难性遗忘，TAME引入回放缓冲区，存储过往任务的代表性样本与嵌入表示，并在训练中重用。注意力机制进一步优先选取对当前预测最相关的历史信息。实验基于从CIFAR-100衍生的二分类任务，结果表明TAME在新任务上显著提升准确率，同时保持早期任务的性能，验证了其在持续学习中兼顾适应性与知识保留的有效性。
原文摘要 · Abstract (English)
Lifelong deep learning (LDL) trains neural networks to learn sequentially across tasks while preserving prior knowledge. We propose Task-Aware Multi-Expert (TAME), a continual learning algorithm that leverages task similarity to guide expert selection and knowledge transfer. TAME maintains a pool of pretrained neural networks and activates the most relevant expert for each new task. A shared dense layer integrates features from the chosen expert to generate predictions. To reduce catastrophic forgetting, TAME uses a replay buffer that stores representative samples and embeddings from previous tasks and reuses them during training. An attention mechanism further prioritizes the most relevant stored information for each prediction. Together, these components allow TAME to adapt flexibly while retaining important knowledge across evolving task sequences. Experiments on binary classification tasks derived from CIFAR-100 show that TAME improves accuracy on new tasks while sustaining performance on earlier ones, highlighting its effectiveness in balancing adaptation and retention in lifelong learning settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。