arXiv:2605.10723cs.CVcs.AI2026-05

用多智能体协同生成音乐视频,兼顾视觉一致性与成本控制。

AgentMV: A State-Guided Multi-Agent Framework for Budget-Aware Music Video Generation

论文配图:AgentMV: A State-Guided Multi-Agent Framework for Budget-Aware Music Video Generation
图 1 · 摘自论文原文
  • 分角色智能体协作,通过状态追踪实现全程信息共享。
  • 将视频生成建模为带预算约束的多重选择背包问题,优化资源分配。
  • 适合需要高质量长视频生成且关注成本的创作者或平台使用。

从一首歌曲生成完整的音乐视频,不仅需要为每句歌词生成视觉上合理的片段,还需保持长期视觉一致性、协调重复元素、同步剪辑与音乐结构,并管理生成过程的累计成本。现有方法通常独立生成各片段或采用固定策略,难以对整首歌进行全局规划。本文提出AgentMV,一种面向预算感知的多智能体音乐视频生成框架。该框架将制作流程分解为音乐感知、剧本规划、视觉资产提供、片段实现和最终合成等专用智能体,通过结构化持久状态实现全过程中信息传递与状态追踪。为优化资源使用,将动机感知的片段实现建模为组级多重选择背包问题,利用动态规划求解,综合考虑片段重要性、生成质量、成本及动机复用,在全局预算约束下实现最优决策。在歌曲基准上的实验表明,AgentMV在质量-成本权衡上优于现有框架,凸显了状态引导的多智能体协同与预算感知规划在长时音乐视频生成中的潜力。

原文摘要 · Abstract (English)

Generating a complete music video from a song requires more than synthesizing visually plausible clips for individual lyric prompts. A practical system must maintain long-range visual consistency, coordinate recurring motifs, synchronize edits with musical structure, and manage the cumulative cost of video generation. Existing approaches typically generate segments independently or adopt fixed generation strategies, limiting their ability to perform global planning over an entire song. We present AgentMV, a state-guided multi-agent framework for budget-aware music video generation. AgentMV decomposes the production process into specialized agents for music perception, script planning, visual asset provision, segment realization, and final assembly, coordinated through a Structured Persistent State that enables information exchange and state tracking throughout the generation process. To optimize generation resources, we formulate motif-aware segment realization as a group-level Multiple-Choice Knapsack Problem solved via dynamic programming, considering segment importance, generation quality, cost, and motif reuse under a global budget constraint. Experiments on song benchmarks demonstrate that AgentMV improves quality-cost trade-offs over existing MV generation frameworks, highlighting the potential of state-guided multi-agent coordination and budget-aware planning for long-form music video generation with improved quality and efficiency.

音乐视频生成多智能体预算优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。