优化任务顺序可显著提升多任务持续学习效果
Optimal Task Order for Continual Learning of Multiple Tasks
- 按代表性从弱到强排列任务，相邻任务应尽量不同
- 在多个数据集上验证，模型性能普遍提升
- 适用于各类神经网络，具有通用性
持续学习多个任务仍是神经网络的重大挑战。本文研究任务顺序对持续学习的影响，提出一种优化策略。基于带有潜在因子的线性师生模型，推导出任务相似性与排序对学习性能的影响表达式。分析表明，在广泛参数范围内存在两条普适原则：(1) 任务应从代表性最弱到最强排列；(2) 相邻任务应尽量不相似。在合成数据和真实图像分类数据集（Fashion-MNIST、CIFAR-10、CIFAR-100）上验证，无论多层感知机还是卷积神经网络均表现出一致的性能提升。本工作为任务增量持续学习中的任务顺序优化提供通用框架。
原文摘要 · Abstract (English)
Continual learning of multiple tasks remains a major challenge for neural networks. Here, we investigate how task order influences continual learning and propose a strategy for optimizing it. Leveraging a linear teacher-student model with latent factors, we derive an analytical expression relating task similarity and ordering to learning performance. Our analysis reveals two principles that hold under a wide parameter range: (1) tasks should be arranged from the least representative to the most typical, and (2) adjacent tasks should be dissimilar. We validate these rules on both synthetic data and real-world image classification datasets (Fashion-MNIST, CIFAR-10, CIFAR-100), demonstrating consistent performance improvements in both multilayer perceptrons and convolutional neural networks. Our work thus presents a generalizable framework for task-order optimization in task-incremental continual learning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。