通过分层二阶优化缓解任务顺序敏感与遗忘问题
Mitigating Task-Order Sensitivity and Forgetting via Hierarchical Second-Order Consolidation
- 采用赫斯矩阵正则的泰勒展开整合局部更新
- 在多数据集上提升平均准确率7%~25%
- 适合需要稳定连续学习的工业场景
我们提出一种分层泰勒级数持续学习框架（HTCL），将快速局部适应与保守的二阶全局巩固相结合，以缓解随机任务顺序带来的高方差问题。为减少任务顺序影响，HTCL识别组内最优任务序列，并通过赫斯矩阵正则化的泰勒展开整合局部更新，实现具有理论保证的巩固步骤。该方法可自然扩展至L层层级结构，支持传统单层持续学习系统无法实现的多尺度知识融合。在多个数据集及回放与正则化基线中，HTCL作为模型无关的巩固层，始终提升性能，使平均准确率提升7%至25%，同时在随机任务排列下将最终准确率的标准差降低高达68%。
原文摘要 · Abstract (English)
We introduce $\textbf{Hierarchical Taylor Series-based Continual Learning (HTCL)}$, a framework that couples fast local adaptation with conservative, second-order global consolidation to address the high variance introduced by random task ordering. To address task-order effects, HTCL identifies the best intra-group task sequence and integrates the resulting local updates through a Hessian-regularized Taylor expansion, yielding a consolidation step with theoretical guarantees. The approach naturally extends to an $L$-level hierarchy, enabling multiscale knowledge integration in a manner not supported by conventional single-level CL systems. Across a wide range of datasets and replay and regularization baselines, HTCL acts as a model-agnostic consolidation layer that consistently enhances performance, yielding mean accuracy gains of $7\%$ to $25\%$ while reducing the standard deviation of final accuracy by up to $68\%$ across random task permutations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。