提出DRDN模型,让ViT在从零训练下持续学习新类别时更稳定准确。
DRDN: Decoupled Representation Dynamic Network for From-Scratch ViT Class-Incremental Learning

- 用掩码图像建模强化共享主干的通用视觉表征
- 分层任务令牌扩展减少任务间混淆,平均准确率达77.19%
- 无需额外推理开销,适合长期增量学习场景
类增量学习(CIL)中动态扩展方法通过增加专用标记或子网络保护特定任务知识,但我们的分析表明,仅靠分类监督无法长期保持任务无关的共享主干表征。我们识别出两个相互关联的挑战:在以当前任务数据为主顺序训练时产生跨任务混淆,使决策边界偏向近期任务;以及主干中共享表征优化不足,随任务积累限制长期判别能力。为此提出解耦表征动态网络(DRDN),通过两个正交机制应对:对共享主干,每一步增量训练均持续应用掩码图像建模(MIM),重建梯度仅通过主干传递,促使主干保留超越类别判别线索的一般视觉结构;对任务特定判别,采用跨所有Transformer层的分层任务标记扩展,并引入改进的每任务注意力规则,降低任务间干扰。通过准确率下降分析和跨任务混淆率测量验证设计。在无外部预训练的从零训练ViT CIL设置下,DRDN在与基线相当的主干规模下持续优于强基线。在CIFAR100-B0(10步)上达77.19%平均准确率,较DKT高1.36点,较DyTox高3.53点,优势随序列增长;多种子验证确认稳定性(±0.31%)。MIM解码器仅在训练时激活,不增加推理参数或计算量。
原文摘要 · Abstract (English)
Dynamic expansion methods for class-incremental learning (CIL) protect task-specific knowledge by growing dedicated tokens or subnetworks, yet our analyses suggest that classification supervision alone does not sufficiently preserve task-agnostic shared backbone representations over long incremental sequences. We identify two intertwined challenges: cross-task confusion from sequential training on predominantly current-task data, which biases decision boundaries toward recent tasks; and under-optimized shared representations in the backbone that cap long-term discriminability as tasks accumulate. We propose the Decoupled Representation Dynamic Network (DRDN), which addresses these challenges via two orthogonal mechanisms. For shared backbone representations, DRDN continuously applies masked image modeling (MIM) at every incremental step, with reconstruction gradients routed exclusively through the backbone, encouraging it to retain general visual structure beyond class-discriminative cues. For task-specific discrimination, DRDN employs hierarchical task token expansion across all transformer layers, with a modified per-task attention rule that reduces inter-task interference. We support this design with accuracy degradation analysis and cross-task confusion rate measurements. In the from-scratch ViT CIL setting (no external pretraining), DRDN consistently improves over strong token-expansion baselines with comparable backbone scale. On CIFAR100-B0 (10 steps), DRDN achieves 77.19% average accuracy, outperforming DKT by 1.36 points and DyTox by 3.53 points, with an advantage that grows at longer incremental sequences. Multi-seed validation confirms stability (+/-0.31%). The MIM decoder is active only during training, adding no inference-time parameters or computation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。