无需微调，一键生成角色视频，轻松实现可控角色动画。
MovieCharacter: A Tuning-Free Framework for Controllable Character Video Synthesis
- 模块化设计：分步处理角色分割、运动模仿与视频合成。
- 不依赖微调，仅用开源模型即达高质量生成效果。
- 适合内容创作者快速制作角色动画，易用性强。
当前角色视频生成仍需大量微调或复杂的3D建模流程，限制了其可及性与实时应用。为此，我们提出一种简单高效的无微调框架 MovieCharacter，旨在简化生成流程并保证高质量输出。该框架将合成任务分解为角色分割与追踪、视频对象移除、角色运动模仿和视频合成四个独立模块，模块化设计便于灵活定制，且各组件协同工作以满足用户需求。通过利用现有开源模型并集成成熟技术，MovieCharacter 在无需大量资源或专有数据集的情况下，实现了出色的生成效果。实验表明，该框架显著提升了角色视频合成的效率、可及性与适应性，为更广泛的创作与交互应用铺平道路。
原文摘要 · Abstract (English)
Recent advancements in character video synthesis still depend on extensive fine-tuning or complex 3D modeling processes, which can restrict accessibility and hinder real-time applicability. To address these challenges, we propose a simple yet effective tuning-free framework for character video synthesis, named MovieCharacter, designed to streamline the synthesis process while ensuring high-quality outcomes. Our framework decomposes the synthesis task into distinct, manageable modules: character segmentation and tracking, video object removal, character motion imitation, and video composition. This modular design not only facilitates flexible customization but also ensures that each component operates collaboratively to effectively meet user needs. By leveraging existing open-source models and integrating well-established techniques, MovieCharacter achieves impressive synthesis results without necessitating substantial resources or proprietary datasets. Experimental results demonstrate that our framework enhances the efficiency, accessibility, and adaptability of character video synthesis, paving the way for broader creative and interactive applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。