arXiv:2501.07983cs.CV2025-01ECCV被引 1

让视频自动匹配不同风格,智能推荐流畅转场。

V-Trans4Style: Visual Transition Recommendation for Video Production Style Adaptation

  • 用变压器网络从视频中学习连贯转场序列。
  • 在6000段视频上测试,转场推荐准确率提升10%至80%。
  • 可精准适配纪录片、电影等不同制作风格,适合内容创作者。

我们提出V-Trans4Style,一种针对动态视频编辑需求的创新算法,可将视频适配至纪录片、剧情片、电影或特定YouTube频道的制作风格。该方法通过基于Transformer的编码器-解码器网络,仅凭输入视频学习生成时序一致且视觉无缝的转场序列,并引入风格条件模块,利用激活最大化迭代调整解码器输出的转场效果。我们在新发布的AutoTransition++数据集(6000段视频,含多种制作风格类别)上验证了有效性。编码器-解码器模型在Recall@K和均排名指标上相比基线提升10%至80%;风格条件模块使转场更贴近目标风格特征,相似度平均提高约12%。本工作为深入探索视频制作风格提供了基础。

原文摘要 · Abstract (English)

We introduce V-Trans4Style, an innovative algorithm tailored for dynamic video content editing needs. It is designed to adapt videos to different production styles like documentaries, dramas, feature films, or a specific YouTube channel's video-making technique. Our algorithm recommends optimal visual transitions to help achieve this flexibility using a more bottom-up approach. We first employ a transformer-based encoder-decoder network to learn recommending temporally consistent and visually seamless sequences of visual transitions using only the input videos. We then introduce a style conditioning module that leverages this model to iteratively adjust the visual transitions obtained from the decoder through activation maximization. We demonstrate the efficacy of our method through experiments conducted on our newly introduced AutoTransition++ dataset. It is a 6k video version of AutoTransition Dataset that additionally categorizes its videos into different production style categories. Our encoder-decoder model outperforms the state-of-the-art transition recommendation method, achieving improvements of 10% to 80% in Recall@K and mean rank values over baseline. Our style conditioning module results in visual transitions that improve the capture of the desired video production style characteristics by an average of around 12% in comparison to other methods when measured with similarity metrics. We hope that our work serves as a foundation for exploring and understanding video production styles further.

视频生成风格迁移转场推荐Transformer

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。