首个原生音视频联合生成模型，实现精准口型同步与电影级镜头控制。
Seedance 1.5 pro: A Native Audio-Visual Joint Generation Foundation Model
- 采用双分支扩散Transformer架构，融合跨模态联合模块与多阶段数据流水线。
- 推理速度提升10倍以上，支持多语言方言口型同步与叙事连贯性增强。
- 适合影视制作、虚拟人播报等专业内容生成场景，已上线火山引擎平台。
近期视频生成技术的发展推动了统一音视频生成的进程。本文提出Seedance 1.5 pro，一个专为原生音视频联合生成设计的基础模型。该模型基于双分支扩散Transformer架构，结合跨模态联合模块与专用多阶段数据处理流程，在音视频同步性和生成质量上表现卓越。为确保实用性，我们实施了精细的后训练优化，包括在高质量数据集上的监督微调（SFT）以及基于多维奖励模型的人类反馈强化学习（RLHF）。此外，我们引入加速框架，使推理速度提升超过10倍。Seedance 1.5 pro 在多语言及方言口型同步、动态电影级镜头控制和叙事连贯性方面表现出色，可作为专业级内容创作的强大引擎。该模型现已在火山引擎开放使用：https://console.volcengine.com/ark/region:ark+cn-beijing/experience/vision?type=GenVideo。
原文摘要 · Abstract (English)
Recent strides in video generation have paved the way for unified audio-visual generation. In this work, we present Seedance 1.5 pro, a foundational model engineered specifically for native, joint audio-video generation. Leveraging a dual-branch Diffusion Transformer architecture, the model integrates a cross-modal joint module with a specialized multi-stage data pipeline, achieving exceptional audio-visual synchronization and superior generation quality. To ensure practical utility, we implement meticulous post-training optimizations, including Supervised Fine-Tuning (SFT) on high-quality datasets and Reinforcement Learning from Human Feedback (RLHF) with multi-dimensional reward models. Furthermore, we introduce an acceleration framework that boosts inference speed by over 10X. Seedance 1.5 pro distinguishes itself through precise multilingual and dialect lip-syncing, dynamic cinematic camera control, and enhanced narrative coherence, positioning it as a robust engine for professional-grade content creation. Seedance 1.5 pro is now accessible on Volcano Engine at https://console.volcengine.com/ark/region:ark+cn-beijing/experience/vision?type=GenVideo.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。