提出分层双子空间解耦方法,缓解视觉语言模型持续学习中的遗忘问题。
Hierarchical Dual-Subspace Decoupling for Continual Learning in Vision-Language Models

- 将参数空间分解为通用与任务特异子空间,分离不同任务更新路径。
- 通过自适应阈值与SVD分解,减少跨任务子空间干扰,提升知识保留能力。
- 适合需要长期增量学习的视觉语言模型场景,尤其关注稳定性与泛化性。
类增量学习旨在持续获取新知识的同时保留已有信息,从而缓解灾难性遗忘。现有方法主要限制参数更新,但常忽略其在高维空间中的结构特性。从子空间视角看,不同任务引起的参数更新往往位于多个重叠的低秩子空间中,导致跨任务子空间干扰和严重遗忘。为此,我们提出HDSD:一种面向视觉语言模型持续学习的分层双子空间解耦框架。具体而言,设计轻量级特征调制模块(FMM),显式将参数空间分解为通用与任务特异子空间。在此基础上,开发两个互补组件:一是通用融合模块(GFM),通过评估跨任务相对参数变化并使用自适应阈值捕捉稳定可迁移知识;二是分层学习模块(HLM),利用奇异值分解(SVD)进行结构化参数分解,并通过缩放机制将更新约束在不同子空间尺度内。上述设计有效降低子空间干扰与参数漂移。在常规基准上的大量实验表明,HDSD达到当前最优性能。
原文摘要 · Abstract (English)
Class-incremental learning aims to continuously acquire new knowledge while preserving previously learned information, thereby mitigating catastrophic forgetting. Existing methods primarily restrict parameter updates but often overlook their structural properties in high-dimensional spaces. From a subspace perspective, updates induced by different tasks tend to lie in multiple overlapping low-rank subspaces, leading to cross-task subspace interference and severe forgetting. To address this issue, we propose HDSD, a Hierarchical Dual-Subspace Decoupling framework for continual learning in vision-language models. Specifically, we introduce a lightweight Feature Modulation Module (FMM) that explicitly decomposes the parameter space into general and task-specific subspaces. Building on this design, we develop two complementary components. First, a General Fusion Module (GFM) evaluates relative parameter changes across tasks and uses an adaptive threshold to capture stable and transferable knowledge. Second, a Hierarchical Learning Module (HLM) performs structured parameter decomposition via Singular Value Decomposition (SVD) and uses a scaling mechanism to constrain updates within distinct subspace scales. Together, these designs reduce subspace interference and parameter drift. Extensive experiments on conventional benchmarks show that HDSD achieves state-of-the-art results.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。