arXiv:2508.00144cs.CV2025-08被引 5

提出视频生成质量评估新指标,衡量内容一致性

World Consistency Score: A Unified Metric for Video Generation Quality

  • 设计四项可解释子指标,评估物体持续性、关系稳定性等
  • 通过人类偏好数据训练加权公式,得分与人评高度一致
  • 适合评估长时序视频生成模型,尤其关注世界连贯性

我们提出世界一致性评分(World Consistency Score, WCS),一种新型统一视频生成质量评估指标,强调生成视频的内部世界一致性。WCS整合四个可解释的子指标:物体持续性、关系稳定性、因果合规性与闪烁惩罚,分别衡量视频在时间与物理层面的连贯性。这些子指标通过学习得到的加权公式融合为单一一致性分数,与人类判断高度一致。本文阐述了现有视频评估指标的局限性,形式化定义各子指标并说明其计算方法(使用开源工具如追踪器、动作识别器、CLIP嵌入、光流),并基于人类偏好数据训练权重。实验验证蓝图包括在VBench-2.0、EvalCrafter、LOVE等基准上测试WCS与人类评价的相关性,进行敏感性分析,并与FVD、CLIPScore、VBench、FVMD等主流指标对比。WCS提供了一个全面且可解释的框架,用于评估生成模型维持长时间连贯‘世界’的能力,弥补了以往仅关注视觉保真度或提示对齐的不足。

原文摘要 · Abstract (English)

We introduce World Consistency Score (WCS), a novel unified evaluation metric for generative video models that emphasizes internal world consistency of the generated videos. WCS integrates four interpretable sub-components - object permanence, relation stability, causal compliance, and flicker penalty - each measuring a distinct aspect of temporal and physical coherence in a video. These submetrics are combined via a learned weighted formula to produce a single consistency score that aligns with human judgments. We detail the motivation for WCS in the context of existing video evaluation metrics, formalize each submetric and how it is computed with open-source tools (trackers, action recognizers, CLIP embeddings, optical flow), and describe how the weights of the WCS combination are trained using human preference data. We also outline an experimental validation blueprint: using benchmarks like VBench-2.0, EvalCrafter, and LOVE to test WCS's correlation with human evaluations, performing sensitivity analyses, and comparing WCS against established metrics (FVD, CLIPScore, VBench, FVMD). The proposed WCS offers a comprehensive and interpretable framework for evaluating video generation models on their ability to maintain a coherent "world" over time, addressing gaps left by prior metrics focused only on visual fidelity or prompt alignment.

视频生成评估指标一致性可解释性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。