首个从人类感知出发评估视频运动质量的基准,解决现有方法与人眼判断不符的问题。
VMBench: A Benchmark for Perception-Aligned Video Motion Generation
- 基于人类感知设计五维运动评估指标,更精准反映生成视频的运动质量。
- 利用大模型自动生成多样化运动提示,覆盖六类动态场景,提升测试多样性。
- 通过真人偏好标注验证,指标相关性比基线提升35.3%,适合研究高质量运动生成者。
视频生成技术迅速发展,但视频运动质量的评估仍是难题。现有方法存在两大问题:一是运动评价指标与人类感知不一致;二是运动提示种类有限。为此,我们提出VMBench——首个以人类感知对齐为核心的视频运动基准。该基准具备三大特性:1)感知驱动的运动评估指标,基于人类对运动视频的五维感知维度,构建细粒度评价体系,深入揭示模型在运动质量上的优劣;2)元信息引导的运动提示生成,通过提取元信息,借助大模型生成多样化运动提示,并经人机协同验证,形成涵盖六类动态场景的多层级提示库;3)人机对齐的验证机制,提供真人偏好标注,使我们的评估指标在斯皮尔曼相关性上相较基线平均提升35.3%。这是首次从人类感知角度系统评估视频运动质量。VMBench将很快在https://github.com/GD-AIGC/VMBench发布,为运动生成模型的评测与进步树立新标准。
原文摘要 · Abstract (English)
Video generation has advanced rapidly, improving evaluation methods, yet assessing video's motion remains a major challenge. Specifically, there are two key issues: 1) current motion metrics do not fully align with human perceptions; 2) the existing motion prompts are limited. Based on these findings, we introduce VMBench--a comprehensive Video Motion Benchmark that has perception-aligned motion metrics and features the most diverse types of motion. VMBench has several appealing properties: 1) Perception-Driven Motion Evaluation Metrics, we identify five dimensions based on human perception in motion video assessment and develop fine-grained evaluation metrics, providing deeper insights into models' strengths and weaknesses in motion quality. 2) Meta-Guided Motion Prompt Generation, a structured method that extracts meta-information, generates diverse motion prompts with LLMs, and refines them through human-AI validation, resulting in a multi-level prompt library covering six key dynamic scene dimensions. 3) Human-Aligned Validation Mechanism, we provide human preference annotations to validate our benchmarks, with our metrics achieving an average 35.3% improvement in Spearman's correlation over baseline methods. This is the first time that the quality of motion in videos has been evaluated from the perspective of human perception alignment. Additionally, we will soon release VMBench at https://github.com/GD-AIGC/VMBench, setting a new standard for evaluating and advancing motion generation models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。