arXiv:2606.01703cs.SDcs.AI2026-06被引 1

让AI为长视频自动生成连贯过渡的音乐,像专业作曲家一样懂剧情变化。

JenBridge: Adaptive Long-Form Video Soundtracking across Scene Transitions

论文配图:JenBridge: Adaptive Long-Form Video Soundtracking across Scene Transitions
图 1 · 摘自论文原文
  • 用双模态条件+流匹配训练,让音乐精准贴合视频内容
  • 通过智能选型机制,使场景切换时音乐自然过渡
  • 专设评测基准,量化评估音乐与剧情衔接质量

针对长视频配乐中跨场景转换缺乏连贯性的问题,本文提出JenBridge框架,实现高保真、长时序且叙事一致的自动配乐。核心采用基于Transformer的生成模型,以流匹配为目标,在大规模文本-音频语料上预训练建立音乐先验,再通过文本与视觉双重条件微调实现跨模态对齐。为保障长序列一致性,引入创新自适应过渡机制:集成多种过渡风格,结合大型语言模型(LLM)代理作为导演,智能选择每处剧情转折的最佳音乐过渡方式。为此,我们构建了全新的LVS基准,包含精选数据集与聚焦整体性与过渡感知的评价指标。在该基准上的大量实验表明,JenBridge在客观与主观评价上均显著优于现有方法,尤其在场景过渡自然度与整体叙事连贯性方面表现突出。JenBridge标志着向全自动、专业级视频配乐迈出了关键一步。

原文摘要 · Abstract (English)

We address the challenge of generating high-fidelity, long-form soundtracks that remain coherent across scene transitions. Existing AI music systems are mainly designed for short, isolated clips and lack mechanisms to ensure narrative continuity. We present JenBridge, a modular and interpretable framework for adaptive long-form video soundtracking that ensures both high-fidelity audio generation and transition naturalness. The core architecture is a Transformer-based generative model trained with a flow-matching objective, following a two-stage paradigm: pretraining on large-scale text-audio corpora to establish robust musical priors, then adapting to the video domain with dual text-visual conditioning for precise cross-modal alignment. Crucially, to achieve long-form coherence across diverse scene changes, JenBridge incorporates a novel adaptive transition mechanism. This system features a versatile toolkit of transition styles, including a generative transition method, and uniquely employs a Large Language Model (LLM) Agent that acts as a director to select the most appropriate transition for each narrative shift intelligently. To rigorously assess this task, we propose the LVS Benchmark, a new benchmark that includes a curated dataset and novel evaluation metrics focusing on holistic and transition-aware assessment. Extensive experiments on the proposed benchmark demonstrate that JenBridge significantly outperforms existing methods in both objective and subjective metrics, particularly in terms of transition naturalness and overall narrative coherence. JenBridge represents a significant step towards fully automated, professional-quality video soundtracking.

视频配乐长视频过渡生成LLM应用

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。