用智能路由系统自动选择最佳模型,提升视频质量评估的通用性与可解释性。
Q-Router: Agentic Video Quality Assessment with Expert Model Routing and Artifact Localization
- 基于视觉语言模型动态调度多个专家模型,按视频内容智能选型。
- 在多个基准上超越现有模型,尤其在AIGC和短视频场景下表现更优。
- 可定位视频中的时空伪影,适合用于生成模型的奖励函数设计。
视频质量评估(VQA)是计算机视觉的基础任务,旨在预测视频的主观质量。现有高性能模型依赖直接评分监督,存在跨内容类型(如UGC、短视频、AIGC)泛化能力差、可解释性弱、难以扩展至新场景等问题。本文提出Q-Router,一种面向通用VQA的代理式框架,采用多层级模型路由机制。该框架集成多种专家模型,并利用视觉-语言模型(VLMs)作为实时路由器,根据输入视频语义动态推理并融合最适专家。路由系统按计算预算分层,最重层级包含时空伪影定位,增强可解释性。该代理设计结合了专业模型的互补优势,在异构视频源和任务中实现一致性能。大量实验表明,Q-Router在多个基准上达到或超过当前最优水平,显著提升泛化能力和可解释性。此外,其在质量问答基准Q-Bench-Video上表现优异,显示出作为下一代VQA基础系统的潜力。最后,我们证明Q-Router能有效定位时空伪影,具备作为视频生成模型后训练奖励函数的前景。
原文摘要 · Abstract (English)
Video quality assessment (VQA) is a fundamental computer vision task that aims to predict the perceptual quality of a given video in alignment with human judgments. Existing performant VQA models trained with direct score supervision suffer from (1) poor generalization across diverse content and tasks, ranging from user-generated content (UGC), short-form videos, to AI-generated content (AIGC), (2) limited interpretability, and (3) lack of extensibility to novel use cases or content types. We propose Q-Router, an agentic framework for universal VQA with a multi-tier model routing system. Q-Router integrates a diverse set of expert models and employs vision--language models (VLMs) as real-time routers that dynamically reason and then ensemble the most appropriate experts conditioned on the input video semantics. We build a multi-tiered routing system based on the computing budget, with the heaviest tier involving a specific spatiotemporal artifacts localization for interpretability. This agentic design enables Q-Router to combine the complementary strengths of specialized experts, achieving both flexibility and robustness in delivering consistent performance across heterogeneous video sources and tasks. Extensive experiments demonstrate that Q-Router matches or surpasses state-of-the-art VQA models on a variety of benchmarks, while substantially improving generalization and interpretability. Moreover, Q-Router excels on the quality-based question answering benchmark, Q-Bench-Video, highlighting its promise as a foundation for next-generation VQA systems. Finally, we show that Q-Router capably localizes spatiotemporal artifacts, showing potential as a reward function for post-training video generation models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。