arXiv:2607.18716cs.CV2026-07中稿 · ACM MM 2026

让视频多模态模型持续学习新领域,不遗忘旧知识。

Continual Video-MLLM Adaptation over Evolving Domains

论文配图:Continual Video-MLLM Adaptation over Evolving Domains
图 1 · 摘自论文原文
  • 用轻量专家隔离不同领域,避免知识干扰。
  • 在10个视频问答数据集上提升性能,显著减少遗忘。
  • 适合长期部署的视频理解系统,尤其关注持续学习。

视频多模态大模型在视频理解方面表现强劲,但其对连续演进领域的适应能力仍待探索。现实应用中,视频数据来自异构领域且持续流入,模型需在不覆盖已有知识的前提下学习新领域知识。现有持续学习方法依赖共享适配空间,易引发跨域干扰和灾难性遗忘。本文提出分布感知专家路由(DAER),一种参数高效框架,用于在不断演进的领域上持续适配视频-多模态大模型。DAER保持领域隔离的轻量级专家,同时冻结预训练模型主干,实现领域特化与通用多模态知识的解耦。引入基于最大均值差异(MMD)的领域内分布感知路由机制,将输入匹配至专家级原型库。针对推理时无任务标识的问题,设计判别子空间中的跨域路由机制以增强领域识别鲁棒性。此外,提出自适应领域合并策略提升参数可扩展性,并采用两阶段优化稳定专家专属性。我们在涵盖10个不同视觉环境与推理需求的VidQA数据集上构建了领域增量基准进行评估。实验在两个强基线视频-多模态大模型上验证,结果表明DAER持续优于现有方法。

原文摘要 · Abstract (English)

Video multimodal large language models have shown strong capability in video understanding, yet their adaptation to sequentially evolving domains remains underexplored. In real-world deployments, video data often arrives continuously from heterogeneous domains, requiring the model to acquire new domain-specific knowledge without overwriting previously learned capabilities. Existing continual learning methods typically rely on shared adaptation spaces, which can induce severe cross-domain interference and catastrophic forgetting. We propose Distribution-Aware Expert Routing, a parameter-efficient framework for continual Video-MLLM adaptation over evolving domains. DAER maintains domain-isolated lightweight experts while keeping the pretrained Video-MLLM backbone frozen, thereby decoupling domain-specific adaptation from the general multimodal knowledge of the pretrained model. To enable fine-grained specialization, we introduce an intra-domain distribution-aware routing mechanism that matches each input to expert-level prototype reservoirs using MMD. To address the absence of task identities at inference time, we further propose an inter-domain routing mechanism that performs prototype matching in a discriminative subspace for robust domain identification. In addition, we introduce adaptive domain merging to improve parameter scalability and adopt a two-stage optimization strategy to stabilize expert specialization during continual learning. We evaluate DAER by curating a domain-incremental benchmark built from ten VidQA datasets covering diverse visual environments and reasoning demands. Experiments on two strong Video-MLLM backbones show that DAER consistently outperforms prior methods.

视频理解持续学习多模态专家路由

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。