arXiv:2505.01450cs.LG2025-05中稿 · CVPR被引 1

构建电影配音评估基准,提升对白、旁白、独白与演员适配的评测能力

Towards Film-Making Production Dialogue, Narration, Monologue Adaptive Moving Dubbing Benchmarks

  • 设计覆盖对话、旁白、独白与演员适配的多维度评估框架
  • 支持顶尖电影配音模型与多模态大模型的全面评测
  • 开源全部数据与工具,推动电影配音技术持续发展

电影配音技术已取得显著进展,但评估其在真实场景中的表现仍具挑战。建立全面的评估基准对两个关键问题至关重要:1)现有指标无法充分捕捉电影配音中对白、旁白、独白及演员适配的复杂性;2)实用评估系统应能为提升配音质量与电影制作进步提供有效洞察。为此,我们提出适应性配音评估基准(TA-Dubbing),旨在通过适配对白、旁白、独白与演员特性来改进电影制作。TA-Dubbing具有三大优势:1)多维覆盖:涵盖电影理解与语音生成的多种评估维度;2)灵活评测:适用于当前最先进的电影配音模型及先进多模态大语言模型;3)全开源:项目在https://github.com/woka-0a/DeepDubber-V1 全面开源,包含所有视频样本、评估方法与标注数据,并持续将新模型纳入TA-Dubbing排行榜,以推动电影配音领域发展。

原文摘要 · Abstract (English)

Movie dubbing has advanced significantly, yet assessing the real-world effectiveness of these models remains challenging. A comprehensive evaluation benchmark is crucial for two key reasons: 1) Existing metrics fail to fully capture the complexities of dialogue, narration, monologue, and actor adaptability in movie dubbing. 2) A practical evaluation system should offer valuable insights to improve movie dubbing quality and advancement in film production. To this end, we introduce Talking Adaptive Dubbing Benchmarks (TA-Dubbing), designed to improve film production by adapting to dialogue, narration, monologue, and actors in movie dubbing. TA-Dubbing offers several key advantages: 1) Comprehensive Dimensions: TA-Dubbing covers a variety of dimensions of movie dubbing, incorporating metric evaluations for both movie understanding and speech generation. 2) Versatile Benchmarking: TA-Dubbing is designed to evaluate state-of-the-art movie dubbing models and advanced multi-modal large language models. 3) Full Open-Sourcing: We fully open-source TA-Dubbing at https://github.com/woka- 0a/DeepDubber- V1 including all video suits, evaluation methods, annotations. We also continuously integrate new movie dubbing models into the TA-Dubbing leaderboard at https://github.com/woka- 0a/DeepDubber-V1 to drive forward the field of movie dubbing.

电影配音评估基准多模态

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。