arXiv:2411.13609cs.CV2024-11被引 7

提出新视频质量评估方法,兼顾视觉真实与物理合理性。

What You See Is What Matters: A Novel Visual and Physics-Based Metric for Evaluating Video Generation Quality

  • 基于视觉外观与运动合理性设计双模块评分机制
  • 在真实视频扰动测试中,得分与破坏程度高度相关
  • 相比传统指标,更贴近人类对视频质量的判断

随着视频生成模型快速发展,评估生成视频的质量变得日益关键。现有指标如弗雷歇视频距离(FVD)、Inception Score(IS)和ClipSim主要在隐空间衡量质量,忽视了从人类视觉角度出发的外观与运动一致性,以及是否符合物理规律。本文提出一种新指标VAMP(Visual Appearance and Motion Plausibility),评估生成视频的视觉外观与物理合理性。VAMP由两个部分组成:外观分数,评估帧间颜色、形状、纹理的一致性;运动分数,评估物体运动的真实性。通过两项实验验证:一是对真实视频引入多种扰动,测量扰动严重程度与VAMP得分的相关性;二是使用前沿生成模型基于精心设计的提示生成视频,将VAMP结果与人工评价排序对比。结果表明,VAMP能有效捕捉视觉保真度与时间一致性,比传统方法提供更全面的视频质量评估。

原文摘要 · Abstract (English)

As video generation models advance rapidly, assessing the quality of generated videos has become increasingly critical. Existing metrics, such as Fréchet Video Distance (FVD), Inception Score (IS), and ClipSim, measure quality primarily in latent space rather than from a human visual perspective, often overlooking key aspects like appearance and motion consistency to physical laws. In this paper, we propose a novel metric, VAMP (Visual Appearance and Motion Plausibility), that evaluates both the visual appearance and physical plausibility of generated videos. VAMP is composed of two main components: an appearance score, which assesses color, shape, and texture consistency across frames, and a motion score, which evaluates the realism of object movements. We validate VAMP through two experiments: corrupted video evaluation and generated video evaluation. In the corrupted video evaluation, we introduce various types of corruptions into real videos and measure the correlation between corruption severity and VAMP scores. In the generated video evaluation, we use state-of-the-art models to generate videos from carefully designed prompts and compare VAMP's performance to human evaluators' rankings. Our results demonstrate that VAMP effectively captures both visual fidelity and temporal consistency, offering a more comprehensive evaluation of video quality than traditional methods.

视频生成质量评估物理合理性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。