用可学习原型动态生成分类头，缓解持续学习中的灾难性遗忘。
Prototype Augmented Hypernetworks for Continual Learning
- 通过任务原型控制超网络，按需生成特定分类头。
- 在Split-CIFAR100上达74.5%准确率，遗忘仅1.7%。
- 无需存储样本或分类头，适合资源受限场景。
持续学习旨在不遗忘旧知识的前提下顺序学习多个任务，但新任务的梯度更新常覆盖先前学习的权重，导致灾难性遗忘（CF）。本文提出原型增强型超网络（PAH），其单一超网络通过可学习的任务原型动态生成任务特定的分类头。为缓解遗忘，PAH结合交叉熵与双蒸馏损失：一者对齐输出逻辑值，另一者对齐原型，确保跨任务特征表示稳定。在Split-CIFAR100和TinyImageNet上的评估表明，PAH达到当前最优性能，在前者上准确率达74.5%，遗忘率仅1.7%；后者上准确率为63.7%，遗忘率为4.4%，且无需存储样本或分类头。
原文摘要 · Abstract (English)
Continual learning (CL) aims to learn a sequence of tasks without forgetting prior knowledge, but gradient updates for a new task often overwrite the weights learned earlier, causing catastrophic forgetting (CF). We propose Prototype-Augmented Hypernetworks (PAH), a framework where a single hypernetwork, conditioned on learnable task prototypes, dynamically generates task-specific classifier heads on demand. To mitigate forgetting, PAH combines cross-entropy with dual distillation losses, one to align logits and another to align prototypes, ensuring stable feature representations across tasks. Evaluations on Split-CIFAR100 and TinyImageNet demonstrate that PAH achieves state-of-the-art performance, reaching 74.5 % and 63.7 % accuracy with only 1.7 % and 4.4 % forgetting, respectively, surpassing prior methods without storing samples or heads.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。