提出动态注意力机制，让ViT模型实时适应新任务而不遗忘旧知识。
FCL-ViT: Task-Aware Attention Tuning for Continual Learning
- 引入反馈机制生成随任务变化的动态注意力特征。
- 在多个基准上超越现有方法，参数量却保持很小。
- 适合需要持续学习且资源受限的视觉应用。
持续学习（CL）旨在不遗忘旧知识的前提下，将已有深度神经网络（DNN）知识适配到新任务。然而，现有方法多聚焦于为现有DNN添加记忆能力，而非设计能根据任务自适应的新型模型。本文提出新型反馈持续学习视觉变换器（FCL-ViT），通过反馈机制生成实时动态注意力特征，以适应当前任务。FCL-ViT分两阶段运行：第一阶段生成通用图像特征，确定变换器在当前图像上的关注位置；第二阶段生成任务特定图像特征，利用动态注意力。为此，引入可调自注意力模块（TABs）和任务特定模块（TSBs），分别在两阶段中负责调节注意力。FCL-ViT在持续学习任务上超越现有基准方法，同时仅需少量可训练参数。
原文摘要 · Abstract (English)
Continual Learning (CL) involves adapting the prior Deep Neural Network (DNN) knowledge to new tasks, without forgetting the old ones. However, modern CL techniques focus on provisioning memory capabilities to existing DNN models rather than designing new ones that are able to adapt according to the task at hand. This paper presents the novel Feedback Continual Learning Vision Transformer (FCL-ViT) that uses a feedback mechanism to generate real-time dynamic attention features tailored to the current task. The FCL-ViT operates in two Phases. In phase 1, the generic image features are produced and determine where the Transformer should attend on the current image. In phase 2, task-specific image features are generated that leverage dynamic attention. To this end, Tunable self-Attention Blocks (TABs) and Task Specific Blocks (TSBs) are introduced that operate in both phases and are responsible for tuning the TABs attention, respectively. The FCL-ViT surpasses state-of-the-art performance on Continual Learning compared to benchmark methods, while retaining a small number of trainable DNN parameters.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。