arXiv:2608.24763cs.CVcs.LG2026-08中稿 · BMVC 2026

为多任务视频理解设计任务专用专家路由,提升效率与可解释性。

MoTE: Mixture of Task Experts for Multi-Task Video Understanding

论文配图:MoTE: Mixture of Task Experts for Multi-Task Video Understanding
图 1 · 摘自论文原文
  • 将语言模型前馈网络拆分为任务专属专家,按任务样本级路由
  • 五专家模型每样本激活约20亿参数,平均准确率优于现有基线
  • 适合需要高效多任务推理的视频理解场景

流程型视频-语言模型需从同一视觉证据中解决异构任务,如动作识别、预测和流程推断。密集的Transformer解码器在各任务间共享相同前馈网络,易导致任务行为混淆,难以控制能力扩展。稀疏的专家混合(MoE)解码器虽支持条件计算,但基于标记的动态路由与任务级流程目标不自然对齐。本文提出MoTE(任务专家混合),将大语言模型的前馈网络转化为任务专用专家,同时保持多模态主干共享。每个样本遵循样本级任务路径,使活跃任务专家计算独立于存储专家数量。我们以VideoLLM-MoTE实现该设计,在五个COIN基准上使用显式任务路径进行评估。五专家模型每样本激活约20亿语言模型参数,平均顶1准确率高于近期VideoLLM基线。在相同专家拓扑下,优于全专家激活的密集结构和学习的稀疏路由控制。结果表明,任务结构化路由为多任务视频-语言学习提供了可解释且计算高效的解码器替代方案。

原文摘要 · Abstract (English)

Procedural video-language models must solve heterogeneous tasks from the same visual evidence, including action recognition, forecasting, and procedure prediction. Dense transformer decoders share the same feed-forward networks across tasks, which can entangle task behavior and make controlled capability expansion difficult. Sparse Mixture-of-Experts (MoE) decoders provide conditional computation, but token-level learned routing is not naturally aligned with task-level procedural objectives. We propose MoTE (Mixture of Task Experts), a decoder architecture that converts large language model feed-forward networks into task-specific experts while keeping the multimodal backbone shared. Each example follows one sample-level task route, so active task-expert computation remains independent of the number of stored task experts. We instantiate this design as VideoLLM-MoTE and evaluate it on five COIN benchmarks using explicit task routes. The five-expert model activates ~2B LLM parameters per sample and achieves higher average top-1 accuracy than recent VideoLLM baselines. Under the same expert topology, it improves over dense all-expert activation and learned sparse-routing controls. These results show that task-structured routing provides an interpretable and compute-efficient decoder alternative for multi-task video-language learning.

视频理解多任务专家混合语言模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。