arXiv:2504.16003cs.CV2025-04ICCV被引 3

用Mamba模型和统一采样法,让视频质量评估更快更省显存。

MVQA: Mamba with Unified Sampling for Efficient Video Quality Assessment

  • 用Mamba替代传统模型,实现线性计算复杂度。
  • 在保持高精度的同时,速度提升2倍,显存降低至1/5。
  • 融合语义与失真信息采样,不增加计算负担。

长时长、高分辨率视频的快速发展使高效视频质量评估成为关键挑战。现有方法主要通过减少模型参数或重采样输入来应对,但轻量级CNN与Transformer难以兼顾效率与性能,因需处理长序列建模。近期状态空间模型(如Mamba)因其与序列长度呈线性复杂度而成为有前景的替代方案。同时,高效VQA高度依赖对长序列的重采样以降低计算成本,但现有方法常忽视关键语义信息保留。本文提出MVQA,一种基于Mamba的高效视频质量评估模型,并引入新颖的统一语义与失真采样(USDS)策略:从低分辨率视频中采样语义密集区域,从原分辨率视频中采样失真细节区域。为避免双输入带来的计算增加,设计预定义掩码融合机制,实现统一采样,同时捕获语义与质量信息而无额外开销。实验表明,搭载USDS的MVQA在性能接近顶尖方法的同时,速度提升2倍,仅需1/5显存。

原文摘要 · Abstract (English)

The rapid growth of long-duration, high-definition videos has made efficient video quality assessment (VQA) a critical challenge. Existing research typically tackles this problem through two main strategies: reducing model parameters and resampling inputs. However, light-weight Convolution Neural Networks (CNN) and Transformers often struggle to balance efficiency with high performance due to the requirement of long-range modeling capabilities. Recently, the state-space model, particularly Mamba, has emerged as a promising alternative, offering linear complexity with respect to sequence length. Meanwhile, efficient VQA heavily depends on resampling long sequences to minimize computational costs, yet current resampling methods are often weak in preserving essential semantic information. In this work, we present MVQA, a Mamba-based model designed for efficient VQA along with a novel Unified Semantic and Distortion Sampling (USDS) approach. USDS combines semantic patch sampling from low-resolution videos and distortion patch sampling from original-resolution videos. The former captures semantically dense regions, while the latter retains critical distortion details. To prevent computation increase from dual inputs, we propose a fusion mechanism using pre-defined masks, enabling a unified sampling strategy that captures both semantic and quality information without additional computational burden. Experiments show that the proposed MVQA, equipped with USDS, achieve comparable performance to state-of-the-art methods while being $2\times$ as fast and requiring only $1/5$ GPU memory.

视频质量评估Mamba高效模型采样策略

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。