arXiv:2503.14064cs.CV2025-03ICCV被引 6

打造统一视频生成评估工具,解决评测标准分散难题

AIGVE-Tool: AI-Generated Video Evaluation Toolkit with Multifaceted Benchmark

  • 构建五类分类体系,整合多种评估方法
  • 基于5个顶尖模型生成数据集,覆盖9项质量维度
  • 模块化设计适配不同需求,助力下一代视频生成

AI生成视频技术的快速发展催生了对标准化、高效评估指标的迫切需求。现有评估方法缺乏统一框架,难以系统分类,且实现碎片化、接口不统一,造成重复计算开销。同时,多数方法依赖特定数据集,适用性受限。为此,我们提出AIGVE-Tool(AI生成视频评估工具包),一个结构化、可扩展的评估流水线,采用创新的五类分类体系,集成多种评估方法,并通过模块化配置实现灵活定制。此外,我们构建了AIGVE-Bench大型基准数据集,基于五个主流视频生成模型,使用人工设计指令与提示生成内容,系统评估各类模型在九个关键质量维度的表现。大量实验验证了AIGVE-Tool在提供标准化、可靠评估结果方面的有效性,揭示了当前模型的具体优劣势,推动下一代AI生成视频技术发展。

原文摘要 · Abstract (English)

The rapid advancement in AI-generated video synthesis has led to a growth demand for standardized and effective evaluation metrics. Existing metrics lack a unified framework for systematically categorizing methodologies, limiting a holistic understanding of the evaluation landscape. Additionally, fragmented implementations and the absence of standardized interfaces lead to redundant processing overhead. Furthermore, many prior approaches are constrained by dataset-specific dependencies, limiting their applicability across diverse video domains. To address these challenges, we introduce AIGVE-Tool (AI-Generated Video Evaluation Toolkit), a unified framework that provides a structured and extensible evaluation pipeline for a comprehensive AI-generated video evaluation. Organized within a novel five-category taxonomy, AIGVE-Tool integrates multiple evaluation methodologies while allowing flexible customization through a modular configuration system. Additionally, we propose AIGVE-Bench, a large-scale benchmark dataset created with five SOTA video generation models based on hand-crafted instructions and prompts. This dataset systematically evaluates various video generation models across nine critical quality dimensions. Extensive experiments demonstrate the effectiveness of AIGVE-Tool in providing standardized and reliable evaluation results, highlighting specific strengths and limitations of current models and facilitating the advancements of next-generation AI-generated video techniques.

视频评估生成模型基准测试工具包

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。