136亿参数模型实现分钟级高质量长视频生成
LongCat-Video Technical Report
- 统一架构支持文生视频、图生视频与视频续写
- 生成720p/30fps视频仅需数分钟，支持分钟级长视频
- 采用粗到精策略与块稀疏注意力，适合高效视频生成研究
视频生成是构建世界模型的关键路径，高效长视频推理能力尤为重要。为此，我们提出LongCat-Video，一个拥有13.6B参数的基础视频生成模型，在多个视频生成任务中表现优异，尤其在高效且高质量的长视频生成方面表现出色，标志着向世界模型迈出的第一步。其核心特性包括：统一架构支持多任务，基于Diffusion Transformer（DiT）框架，可同时处理文本到视频、图像到视频和视频续写任务；长视频生成能力，通过视频续写预训练，使模型在生成分钟级视频时保持高质量与时间连贯性；高效推理，采用时空双路径粗到精生成策略，可在数分钟内生成720p、30fps视频；块稀疏注意力进一步提升高分辨率下的效率；结合多奖励强化学习人类反馈（Multi-reward RLHF）训练，性能达到最新闭源及领先开源模型水平。代码与模型权重已公开，以加速领域进展。
原文摘要 · Abstract (English)
Video generation is a critical pathway toward world models, with efficient long video inference as a key capability. Toward this end, we introduce LongCat-Video, a foundational video generation model with 13.6B parameters, delivering strong performance across multiple video generation tasks. It particularly excels in efficient and high-quality long video generation, representing our first step toward world models. Key features include: Unified architecture for multiple tasks: Built on the Diffusion Transformer (DiT) framework, LongCat-Video supports Text-to-Video, Image-to-Video, and Video-Continuation tasks with a single model; Long video generation: Pretraining on Video-Continuation tasks enables LongCat-Video to maintain high quality and temporal coherence in the generation of minutes-long videos; Efficient inference: LongCat-Video generates 720p, 30fps videos within minutes by employing a coarse-to-fine generation strategy along both the temporal and spatial axes. Block Sparse Attention further enhances efficiency, particularly at high resolutions; Strong performance with multi-reward RLHF: Multi-reward RLHF training enables LongCat-Video to achieve performance on par with the latest closed-source and leading open-source models. Code and model weights are publicly available to accelerate progress in the field.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。