首个音视频联合编辑数据集，支持指令驱动的多模态视频修改。
JAVEDIT: Joint Audio-Visual Instruction-Guided Video Editing with Agentic Data Curation
- 构建10万条音视频编辑三元组，含五类人类中心视频编辑任务。
- 提出首个标准化评估基准JAVEditBench，覆盖所有编辑类别。
- 首次实现指令引导的音视频联合编辑，性能优于现有方法。
尽管基于指令的视频编辑已取得显著进展，但音视频联合编辑仍受限于缺乏专用数据集与评测基准。为此，我们提出JAVEdit-100k，首个大规模、高质量的指令引导音视频联合编辑数据集。聚焦人类中心视频，该数据集包含约10万条编辑三元组，涵盖主体编辑与语音编辑等五类任务。数据通过四个精心设计的生成流程构建，并结合智能体闭环质量控制机制。此外，为解决领域内评估标准缺失问题，我们引入JAVEditBench，一个涵盖各类编辑任务的综合评测基准，包含经人工对齐的指令与精选源视频。最后，我们提出JAVEdit模型，作为首个指令引导的音视频联合编辑基线模型。实验表明，该模型在六项指标中的五项上超越所有基线。
原文摘要 · Abstract (English)
While instruction-based video editing has seen significant progress, joint audio-visual editing remains constrained by the absence of dedicated datasets and benchmarks. To bridge this gap, we present JAVEdit-100k, the first large-scale, high-quality dataset tailored for instruction-guided joint audio-visual editing. Focusing on human-centric videos, JAVEdit-100k comprises approximately 100K editing triplets spanning five distinct categories, including subject editing and speech editing. This dataset is rigorously constructed via four meticulously designed generation pipelines, seamlessly paired with an agent-in-the-loop quality control mechanism. Furthermore, to address the lack of standardized evaluation within the field, we introduce JAVEditBench, a comprehensive benchmark featuring curated source videos and human-aligned instructions across all editing categories. Finally, we propose JAVEdit, a pioneering baseline model for instruction-guided joint audio-visual editing. Experiments show that \model\ outperforms all baselines on five of six evaluation metrics.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。