通过动态调节梯度更新强度,提升长尾增量学习中旧类和新类的性能。
NeuroGuard: Neural Gradient Update Aware of Representation Damage

- 根据教师预测置信度自适应调整梯度尺度,控制特征更新强度。
- 在五个设置中均超越基线,在四项主流评测中达到最优任务无关准确率。
- 适合关注类别不平衡下模型稳定性与泛化能力的研究者。
长尾增量学习(LT-CIL)需在不均衡数据流中学习新类别同时保留旧类别知识。现有方法主要聚焦于重放机制、分类器或损失函数的改进。本文关注任务边界处特征表示应被更新的强度问题,提出NeuroGuard——一种无需引入可学习参数的更新控制方法,兼容DGR基线的重放记忆、分类器及损失项。自适应梯度缩放(AGS)将教师不确定性转化为任务级梯度缩放因子;置信度排序的知识蒸馏重加权(CRK)赋予教师预测较模糊的重放样本更大蒸馏权重;脆弱性混合熵门(FBE)将旧记忆泄露信息纳入缩放决策。在五个LT-CIL设置中,NeuroGuard均优于DGR;在四个主基准测试中,其任务无关准确率领先所有对比方法。性能提升同时覆盖旧类与新类,中频类准确率在所有五个设置中持续改善。对照实验表明,收益非来自通用梯度抑制:AGS在全部五组设置中优于固定缩放对照,证明边界特异性缩放优于全局统一缩放。
原文摘要 · Abstract (English)
Long-tailed class-incremental learning (LT-CIL) must learn new classes from imbalanced streams while retaining old classes. Existing methods mainly change replay, classifiers, or losses. We study a different factor, namely how strongly the feature representation should be updated at each task boundary. We propose NeuroGuard, an update-control method added to DGR, a replay-based LT-CIL baseline, without adding learnable parameters. NeuroGuard preserves DGR's replay memory, classifier, and set of loss terms. Adaptive Gradient Scaling (AGS) converts teacher uncertainty into one task-wise gradient scale. Confidence-Ranked Knowledge Distillation Reweighting (CRK) gives larger knowledge-distillation weights to replay samples that the teacher predicts less decisively. Fragility-Blended Entropy Gate (FBE) adds old-memory leakage to the scale decision. Across five LT-CIL settings, NeuroGuard improves over DGR in every setting. In the four main benchmark comparisons, it achieves the best task-agnostic accuracy among the compared methods. The gains extend to both old- and new-class accuracy, while medium-frequency accuracy improves consistently across all five settings. Controlled comparisons show that the gain does not come from generic gradient suppression: AGS outperforms a matched fixed-scale control in all five settings, demonstrating that boundary-specific scaling is more effective than applying the same average scale throughout learning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。