arXiv:2607.14645cs.CV2026-07International Conf…

用自回归模型学习电影语法,自动剪辑原始视频成连贯影片。

Autoregressive Modeling of Film with Applications in Video Montage

论文配图:Autoregressive Modeling of Film with Applications in Video Montage
图 1 · 摘自论文原文
  • 基于大规模电影数据训练自回归Transformer,学习电影剪辑规律。
  • 在镜头排序任务上超越现有最佳方法,用户研究显示剪辑质量显著提升。
  • 适用于自动剪辑、人机协同编辑等场景,无需生成新画面。

本文提出FilmGPT,一种自回归Transformer模型,旨在解决视频蒙太奇问题——将零散的原始未剪辑片段转化为连贯的影视序列。受现代大语言模型语言学习启发,我们在大量电影数据上训练长上下文自回归Transformer,直接从数据中隐式学习电影的'语法规则',而非依赖人工规则。与生成式模型不同,FilmGPT不生成新视频帧;推理时引入受原始素材约束的解码算法,根据学得的统计模式从输入片段中选择最优下一镜头。我们首先在标准镜头序列排序基准上评估该模型的下一镜头预测能力,表现优于先前最先进方法。随后通过用户研究验证其在完整电影剪辑任务中的效果,结果表明基于FilmGPT的剪辑显著优于以往方法。最后,展示了FilmGPT在多种视频蒙太奇应用中的实用性,包括自动视频段落裁剪和人机协同剪辑。

原文摘要 · Abstract (English)

This work introduces FilmGPT, an autoregressive transformer designed to address the challenge of video montage--turning a collection of raw, "unwatchable" footage into coherent cinematic sequences. Inspired by language learning in modern LLMs, we train a long-context autoregressive transformer on a large corpus of movies. The aim is to implicitly capture the "grammar" of film directly from data rather than from hand-coded rules. Unlike other generative models, FilmGPT does not generate any new video frames. Instead, at inference time, we introduce a footage-constrained decoding algorithm to select the best next shot from the input raw footage according to the statistical patterns learned from films. We first evaluate these learned statistics directly by using the FilmGPT autoregressive model for next shot prediction on a standard benchmark of shot sequence ordering, outperforming the previous state of the art. We then evaluate our footage-constrained decoding algorithm on the full film editing task via a user study, and find that our FilmGPT-based editing significantly outperforms previous approaches. Finally, we demonstrate the applicability of FilmGPT to a wide range of applications in video montage, from automatic video segment trimming to human-in-the-loop film editing.

视频剪辑自回归模型电影语法蒙太奇

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。