用强化学习让AI像人一样判断视频质量并解释原因。
VQAThinker: Exploring Generalizable and Explainable Video Quality Assessment via Reinforcement Learning
- 基于大模型和强化学习,模拟人类看视频时的推理过程。
- 在多种视频上表现更好,尤其对陌生类型视频泛化能力强。
- 能给出质量差的原因,适合需要可解释性的应用。
视频质量评估(VQA)旨在客观量化视频质量退化程度,使其与人类视觉感知一致。尽管已有进展,现有VQA模型仍存在两大缺陷:对分布外(OOD)视频泛化能力差,且解释性有限,限制了实际应用。为此,我们提出VQAThinker,一种基于推理的VQA框架,利用大模型与强化学习联合建模视频质量理解与评分,模拟人类感知决策。具体地,采用组相对策略优化(GRPO)算法,在评分监督下进行质量推理,并引入三项特定奖励:(1) 钟形回归奖励,预测误差减小时快速提升,接近真实值后敏感度降低;(2) 成对排序奖励,引导模型正确判断视频对之间的相对质量;(3) 时间一致性奖励,鼓励模型偏好时间连贯的视频而非被扰动版本。大量实验表明,VQAThinker在域内与域外基准上均达领先水平,具备强泛化能力。对质量理解任务的评估也证实其在失真归因与质量描述方面优于现有可解释VQA模型与大模型。结果表明,仅通过评分监督,强化学习即可实现通用且可解释的VQA模型构建。
原文摘要 · Abstract (English)
Video quality assessment (VQA) aims to objectively quantify perceptual quality degradation in alignment with human visual perception. Despite recent advances, existing VQA models still suffer from two critical limitations: \textit{poor generalization to out-of-distribution (OOD) videos} and \textit{limited explainability}, which restrict their applicability in real-world scenarios. To address these challenges, we propose \textbf{VQAThinker}, a reasoning-based VQA framework that leverages large multimodal models (LMMs) with reinforcement learning to jointly model video quality understanding and scoring, emulating human perceptual decision-making. Specifically, we adopt group relative policy optimization (GRPO), a rule-guided reinforcement learning algorithm that enables reasoning over video quality under score-level supervision, and introduce three VQA-specific rewards: (1) a \textbf{bell-shaped regression reward} that increases rapidly as the prediction error decreases and becomes progressively less sensitive near the ground truth; (2) a \textbf{pairwise ranking reward} that guides the model to correctly determine the relative quality between video pairs; and (3) a \textbf{temporal consistency reward} that encourages the model to prefer temporally coherent videos over their perturbed counterparts. Extensive experiments demonstrate that VQAThinker achieves state-of-the-art performance on both in-domain and OOD VQA benchmarks, showing strong generalization for video quality scoring. Furthermore, evaluations on video quality understanding tasks validate its superiority in distortion attribution and quality description compared to existing explainable VQA models and LMMs. These findings demonstrate that reinforcement learning offers an effective pathway toward building generalizable and explainable VQA models solely with score-level supervision.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。