让视觉模型动态适应语义变化,旧知识不丢、新概念能学。
Continual Visual Learning under Evolving Semantic Concept Shift

- 通过语义差异定位受影响区域,选择性重写过时映射
- 在多个数据集上实现92.3%的重写准确率与89.1%的保留准确率
- 适合长期运行的智能系统,如自动驾驶、医疗影像分析
视觉基础模型通常假设输入数据外观变化但任务语义不变。然而,在长期运行的视觉系统中,分类体系、政策和概念定义本身可能演进,导致相同视觉证据需不同解释。本文研究这一现象为演化语义概念漂移,并提出SemReWrite框架,可选择性更新过时的视觉-语义映射,同时保留仍有效的知识。该方法通过表征新旧语义规范间的差异,结合稀疏修订监督定位受影响视觉区域,采用输入相关的低秩重写机制与结构化语义记忆、知识保全及无效决策抑制策略。我们进一步构建EvoShift-Bench,涵盖ImageNet、iNaturalist、CUB-200-2011和DomainNet,包含类别分裂、合并、边界修订、插入、部分重定义、重复出现及混合语义-外观漂移等场景。为显式评估选择性语义修订,引入重写准确率(RA)、保留准确率(PA)、过时保留度(OR)和选择性修订评分(SRS)。实验表明,SemReWrite在学习新语义与保留旧知识之间取得更优平衡,优于提示替换、传统微调、参数高效适配及持续学习策略。
原文摘要 · Abstract (English)
Visual foundation models are commonly adapted under the assumption that the appearance of incoming data may change while the semantic meaning of the prediction task remains fixed. In long-lived visual systems, however, taxonomies, policies, and concept definitions can themselves evolve, causing the same visual evidence to require a different interpretation. We study this setting as evolving semantic concept shift and introduce SemReWrite, a framework for selectively updating obsolete visual--semantic mappings while preserving knowledge that remains valid. SemReWrite represents changes between old and revised semantic specifications, combines semantic discrepancy with sparse revised supervision to localize affected visual regions, and uses an input-dependent low-rank rewriting mechanism together with structured semantic memory, preservation, and obsolete-decision suppression. We further introduce EvoShift-Bench, spanning ImageNet, iNaturalist, CUB-200-2011, and DomainNet, with semantic transitions including class split, merge, boundary revision, insertion, partial redefinition, recurrence, and mixed semantic--appearance shift. To explicitly evaluate selective semantic revision, we introduce Rewrite Accuracy (RA) and Preservation Accuracy (PA) for affected and unaffected regions, respectively, Obsolete Retention (OR) for measuring residual outdated semantic associations, and the Selective Revision Score (SRS), which jointly summarizes rewriting and preservation performance. Experiments show that SemReWrite achieves a stronger balance between learning revised semantics and retaining unaffected knowledge than prompt replacement, conventional fine-tuning, parameter-efficient adaptation, and continual-learning strategies.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。