让SAM模型在新场景下持续学习,不占空间还有效
RegCL: Compact Continual SAM Adaptation for Visual Grounding in Multi-Sensorial Media
- 用增量合并轻量模块,不存旧数据也能学新知识
- 五类不同数据集上表现优于现有方法,保持旧能力同时适应新任务
- 适合资源受限的AR/VR、机器人等实时多模态系统
多感官媒体系统(如AR/VR、远程操作、具身AI)需要在感知环境和应用领域不断变化时仍能可靠进行视觉定位。虽然分割一切模型(SAM)提供了强大的密集分割基础,但在医学图像、伪装场景、阴影主导环境等专用动态领域性能下降。现有持续学习方法常依赖回放数据或增长式专用模块,限制了在演进媒体流水线中的紧凑部署。为此,我们提出RegCL,一种无需回放的持续适应框架,通过增量模型合并将多领域分割知识整合进单一SAM适配器。RegCL通过优化合并模型与领域特定适配器间的预测一致性,并保留紧凑的历史特征统计,合并轻量级适配模块(如LoRA风格的AugModules)。在五个异构分割数据集上的实验表明,RegCL在领域增量学习中实现强记忆保持与适应能力,优于对比的非回放持续学习与合并基线。结果表明,RegCL可作为演化多感官媒体流水线中的紧凑视觉适应组件。代码已公开于:https://github.com/Anderw-S/RegCL
原文摘要 · Abstract (English)
Multi-sensorial media systems, including AR/VR, remote operation, and embodied AI, require visual grounding modules that remain reliable as sensing environments and application domains evolve. The Segment Anything Model (SAM) provides a strong foundation for dense visual segmentation, but its performance degrades on specialized and dynamically arriving domains such as medical imagery, camouflaged scenes, and shadow-dominant environments. Existing continual learning methods often rely on replay data or growing domain-specific modules, limiting compact deployment in evolving media pipelines. To address this issue, we propose RegCL, a non-replay continual adaptation framework that consolidates multi-domain segmentation knowledge into a single SAM adapter through incremental model merging. RegCL merges lightweight adaptation modules, e.g., LoRA-style AugModules, by optimizing prediction consistency between the merged model and domain-specific adapters while carrying forward compact historical feature statistics. Experiments across five heterogeneous segmentation datasets show that RegCL achieves strong retention and adaptation under domain-incremental learning, outperforming competitive non-replay continual learning and merging baselines. These results suggest that RegCL can serve as a compact visual adaptation component for evolving multi-sensorial media pipelines. The code is available at \href{https://github.com/Anderw-S/RegCL}{https://github.com/Anderw-S/RegCL}
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。