用拼接短视频生成长视频数据，提升视频理解模型训练效率
VideoWeave: A Data-Centric Approach for Efficient Video Understanding
- 通过拼接已有短视频生成合成的长视频样本
- 相同算力下，模型在视频问答任务上准确率更高
- 适合资源有限但想提升视频模型性能的研究者
视频-语言模型训练成本高昂，主要因长视频序列处理开销大且标注数据稀缺。本文提出 VideoWeave，一种以数据为中心的方法：从现有数据集中选取短时长、带字幕的视频，通过拼接构建合成的长上下文训练样本。该方法不修改模型结构或优化目标，仅重新组织视频-文本对，在固定计算量下扩展时间多样性。我们系统研究了随机拼接与视觉聚类拼接、字幕增强等不同数据组合策略对下游视频问答性能的影响。在相同计算约束下，使用 VideoWeave 训练的模型表现优于传统微调方式。结果表明，仅通过重构训练数据，即可为视频-语言模型提供简单且可扩展的高效训练路径。代码已公开。
原文摘要 · Abstract (English)
Training video-language models is often prohibitively expensive due to the high cost of processing long frame sequences and the limited availability of annotated long videos. We present VideoWeave, a simple yet effective approach to improve data efficiency by constructing synthetic long-context training samples that splice together short, captioned videos from existing datasets. Rather than modifying model architectures or optimization objectives, VideoWeave reorganizes available video-text pairs to expand temporal diversity within fixed compute. We systematically study how different data composition strategies like random versus visually clustered splicing and caption enrichment affect downstream performance on downstream video question answering. Under identical compute constraints, models trained with VideoWeave achieve higher accuracy than conventional video finetuning. Our results highlight that reorganizing training data, rather than altering architectures, may offer a simple and scalable path for training video-language models. We link our code for all experiments here.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。