arXiv:2504.10358cs.CVcs.AI2025-04被引 25

用细粒度问答+推理模型评估生成视频,更准更可解释。

FingER: Content Aware Fine-grained Evaluation with Reasoning for AI-Generated Videos

  • 自动生成视频中实体的细粒度问题,引导模型推理评分。
  • 在3.3k视频上训练,比现有方法提升11.8%(GenAI-Bench)。
  • 适合需要高精度、可解释性视频评估的场景或研究者。

视频生成技术的进步带来了内容评估的巨大挑战,尤其面对日益复杂的模型生成内容。生成视频中的不一致与缺陷本质复杂,整体评分困难。本文强调将细粒度推理融入视频评估的重要性,提出FingER框架:首先利用大语言模型自动生成5个视角的实体级问题,再通过多模态大模型进行推理并打分,最后加权求和得到整体评分。我们构建了包含约3.3k视频与6万条细粒度问答标注的FingER数据集,每条均有详细理由。进一步探索不同训练策略以激发多模态大模型的推理能力。实验表明,采用分组相对策略优化(GRPO)并结合冷启动策略训练的模型表现最佳,在仅使用3.3k训练视频(为其他方法的1/10)的情况下,于GenAI-Bench上超越现有方法11.8%,于MonetBench上提升5.5%。代码与数据集即将开源。

原文摘要 · Abstract (English)

Recent advances in video generation have posed great challenges in the assessment of AI-generated content, particularly with the emergence of increasingly sophisticated models. The various inconsistencies and defects observed in such videos are inherently complex, making overall scoring notoriously difficult. In this paper, we emphasize the critical importance of integrating fine-grained reasoning into video evaluation, and we propose $\textbf{F}$ing$\textbf{ER}$, a novel entity-level reasoning evaluation framework that first automatically generates $\textbf{F}$ine-grained $\textbf{E}$ntity-level questions, and then answers those questions by a $\textbf{R}$easoning model with scores, which can be subsequently weighted summed to an overall score for different applications. Specifically, we leverage LLMs to derive entity-level questions across five distinct perspectives, which (i) often focus on some specific entities of the content, thereby making answering or scoring much easier by MLLMs, and (ii) are more interpretable. Then we construct a FingER dataset, consisting of approximately 3.3k videos and corresponding 60k fine-grained QA annotations, each with detailed reasons. Based on that, we further investigate various training protocols to best incentivize the reasoning capability of MLLMs for correct answer prediction. Extensive experiments demonstrate that a reasoning model trained using Group Relative Policy Optimization (GRPO) with a cold-start strategy achieves the best performance. Notably, our model surpasses existing methods by a relative margin of $11.8\%$ on GenAI-Bench and $5.5\%$ on MonetBench with only 3.3k training videos, which is at most one-tenth of the training samples utilized by other methods. Our code and dataset will be released soon.

视频评估推理模型生成内容细粒度

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。