用预训练模型生成带透明通道的动态视频，支持游戏特效精准控制。
TransAnimate: Taming Layer Diffusion to Generate RGBA Video
- 融合透明图像生成与视频时序模型，复用已有权重提升效率。
- 生成高质量RGBA视频，支持方向箭头与颜色调控运动效果。
- 构建合成数据集缓解透明视频数据稀缺问题，适合游戏与视觉特效场景。
文本到视频生成模型近年取得显著进展，但生成带透明通道（alpha channel）的RGBA视频仍面临数据稀缺和模型适配复杂等挑战。为此，我们提出TransAnimate框架，将透明图像生成技术与视频生成模块结合，实现动态透明视频的生成。该方法高效复用预训练的文生透明图像模型权重，并与在RGB视频上训练的时间模型及可控性插件相结合，适应可控的RGBA视频生成任务。此外，我们设计了交互式运动引导控制机制，通过方向箭头定义运动轨迹，颜色调整缩放比例，实现对游戏特效的精确直观控制。为缓解数据不足问题，我们构建了一套生成RGBA视频数据集的流水线，整合高质量游戏特效视频、提取的前景对象及合成透明视频。大量实验表明，TransAnimate能生成高质量的RGBA视频，成为游戏与视觉特效领域实用有效的工具。
原文摘要 · Abstract (English)
Text-to-video generative models have made remarkable advancements in recent years. However, generating RGBA videos with alpha channels for transparency and visual effects remains a significant challenge due to the scarcity of suitable datasets and the complexity of adapting existing models for this purpose. To address these limitations, we present TransAnimate, an innovative framework that integrates RGBA image generation techniques with video generation modules, enabling the creation of dynamic and transparent videos. TransAnimate efficiently leverages pre-trained text-to-transparent image model weights and combines them with temporal models and controllability plugins trained on RGB videos, adapting them for controllable RGBA video generation tasks. Additionally, we introduce an interactive motion-guided control mechanism, where directional arrows define movement and colors adjust scaling, offering precise and intuitive control for designing game effects. To further alleviate data scarcity, we have developed a pipeline for creating an RGBA video dataset, incorporating high-quality game effect videos, extracted foreground objects, and synthetic transparent videos. Comprehensive experiments demonstrate that TransAnimate generates high-quality RGBA videos, establishing it as a practical and effective tool for applications in gaming and visual effects.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。