小预算下实现强文本到视频生成,靠的是分层架构设计。
Motif-Video 2B: Technical Report

- 分三模块处理不同任务:融合、表征、细节修复
- 用7倍少参数达83.76%评分,超大模型性能
- 适合资源有限但追求高质量视频生成的研究者
训练强大视频生成模型通常需要海量数据、大参数量和大量算力。本文探讨在极小预算下是否仍能实现高质量文本到视频生成:少于1000万视频片段,且训练时间少于10万小时的H200 GPU。核心观点是:模型容量的组织方式比总量更重要。视频生成中的文本对齐、时序一致性与细节恢复常因共用路径而相互干扰。Motif-Video 2B通过架构分离这些功能,而非依赖规模扩张。模型结合两大创新:一是共享交叉注意力增强长序列控制;二是三段式骨干网络分别处理早期融合、联合表征学习与细节精修。为提升小预算下的效率,采用动态令牌路由与前期特征对齐预训练编码器的训练策略。分析显示,其后期模块具备更清晰的跨帧注意力结构。在VBench评测中,该模型达83.76%,超越Wan2.1 14B,但仅使用其7倍少的参数和更少数据,表明精心的架构专业化与高效训练可缩小甚至超越大模型的质量差距。
原文摘要 · Abstract (English)
Training strong video generation models usually requires massive datasets, large parameter counts, and substantial compute. In this work, we ask whether strong text-to-video quality is possible at a much smaller budget: fewer than 10M clips and less than 100,000 H200 GPU hours. Our core claim is that part of the answer lies in how model capacity is organized, not only in how much of it is used. In video generation, prompt alignment, temporal consistency, and fine-detail recovery can interfere with one another when they are handled through the same pathway. Motif-Video 2B addresses this by separating these roles architecturally, rather than relying on scale alone. The model combines two key ideas. First, Shared Cross-Attention strengthens text control when video token sequences become long. Second, a three-part backbone separates early fusion, joint representation learning, and detail refinement. To make this design effective under a limited compute budget, we pair it with an efficient training recipe based on dynamic token routing and early-phase feature alignment to a frozen pretrained video encoder. Our analysis shows that later blocks develop clearer cross-frame attention structure than standard single-stream baselines. On VBench, Motif-Video~2B reaches 83.76\%, surpassing Wan2.1 14B while using 7$\times$ fewer parameters and substantially less training data. These results suggest that careful architectural specialization, combined with an efficiency-oriented training recipe, can narrow or exceed the quality gap typically associated with much larger video models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。