arXiv:2604.07965cs.CVcs.AI2026-04中稿 · CVPR被引 1

通过分离语义子空间实现视觉语言模型的稳定持续编辑。

DSCA: Dynamic Subspace Concept Alignment for Lifelong VLM Editing

  • 将视觉语言表征分解为正交语义子空间,实现概念隔离
  • 单次编辑成功率98%,1000次连续编辑后仍保持95%以上性能
  • 适合需要长期知识更新且避免遗忘的应用场景

模型编辑旨在不重新训练的情况下更新知识以添加新概念和修改信息。持续编辑任务极具挑战性,易导致先前学习的概念被破坏,尤其对视觉语言模型(VLM)而言,因顺序编辑可能引发推理能力下降和跨模态错位。现有基于门控适配器、激活编辑和参数融合的方法虽缓解了全微调中的灾难性遗忘,但仍运行在共享表示空间中,概念纠缠导致编辑相互干扰。本文提出动态子空间概念对齐(DSCA),通过增量聚类与联合视觉语言表征的PCA,将表征空间分解为一组正交语义子空间,并仅在这些变换后的空间中进行编辑,从算法控制转为结构设计,实现概念天然隔离。手术式编辑由多目标损失函数指导,兼顾任务保真度、编辑局部性和跨模态对齐。基础模型冻结下,本方法单次编辑成功率98%,1000次连续编辑后性能仍超95%,幻觉降低3至5个百分点,在持续指令微调基准上取得最优反向迁移(BWT)分数。大量实验表明,DSCA在多种数据集与基准上实现了持续终身编辑的最先进稳定性与知识保留能力。

原文摘要 · Abstract (English)

Model editing aims to update knowledge to add new concepts and change relevant information without retraining. Lifelong editing is a challenging task, prone to disrupting previously learned concepts, especially for Vision Language Models (VLMs), because sequential edits can lead to degraded reasoning and cross modal misalignment. Existing VLM knowledge editing methods based on gated adapters, activation edits, and parameter merging techniques address catastrophic forgetting seen in full fine tuning; however, they still operate in the shared representation space of the VLM, where concepts are entangled, so edits interfere with other non relevant concepts. We hypothesize that this instability persists because current methods algorithmically control edits via optimization rather than structurally separating knowledge. We introduce Dynamic Subspace Concept Alignment (DSCA) which by design mitigates this limitation by decomposing the representation space into a set of orthogonal semantic subspaces and proposing edits only in those transformed spaces. These subspaces are obtained through incremental clustering and PCA on joint vision language representations. This process structurally isolates concepts, enabling precise, non interfering edits by turning isolation from a soft training objective into an architectural property. The surgical edits are guided by a multi term loss function for maintaining task fidelity, edit locality, and cross modal alignment. With the base model frozen, our method achieves 98 percent single edit success, remains over 95 percent after 1000 sequential edits, lowers hallucination by 3 to 5 percent, and achieves the best backward transfer (BWT) scores on continual instruction tuning benchmarks. Extensive experiments demonstrate DSCA state of the art stability and knowledge retention capability in continual lifelong editing across various datasets and benchmarks.

模型编辑视觉语言模型持续学习知识隔离

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。