让两个人打拳的视频生成更真实,避免身份混淆和动作错乱。
MagicFight: Personalized Martial Arts Combat Video Generation

- 基于游戏引擎自建3D武术对抗数据集,支持双人交互生成。
- 改进现有模型,实现角色身份清晰、动作连贯的对打视频。
- 适合影视特效、游戏动画等需要真人对战视频的场景。
随着通用文本到视频生成的兴起,个性化单人视频生成已取得显著进展,但双人互动尤其是武术对打领域仍属空白。现有单人舞蹈生成模型难以捕捉两人对战的复杂性,常出现身份混淆、肢体异常和动作不匹配等问题。为此,我们提出全新任务:个性化武术对打视频生成,并构建名为MagicFight的系统。由于缺乏合适数据,我们利用Unity游戏引擎构建了定制化数据集KungFu-Fiesta,包含多种3D角色、武术动作与场景,覆盖多样化对抗情境。MagicFight通过优化现有模型策略,生成高保真双人对打视频,有效保持个体身份一致性并确保动作序列流畅自然,为交互式视频内容创作奠定基础。
原文摘要 · Abstract (English)
Amid the surge in generic text-to-video generation, the field of personalized human video generation has witnessed notable advancements, primarily concentrated on single-person scenarios. However, to our knowledge, the domain of two-person interactions, particularly in the context of martial arts combat, remains uncharted. We identify a significant gap: existing models for single-person dancing generation prove insufficient for capturing the subtleties and complexities of two engaged fighters, resulting in challenges such as identity confusion, anomalous limbs, and action mismatches. To address this, we introduce a pioneering new task, Personalized Martial Arts Combat Video Generation. Our approach, MagicFight, is specifically crafted to overcome these hurdles. Given this pioneering task, we face a lack of appropriate datasets. Thus, we generate a bespoke dataset using the game physics engine Unity, meticulously crafting a multitude of 3D characters, martial arts moves, and scenes designed to represent the diversity of combat. MagicFight refines and adapts existing models and strategies to generate high-fidelity two-person combat videos that maintain individual identities and ensure seamless, coherent action sequences, thereby laying the groundwork for future innovations in the realm of interactive video content creation. Website: https://MingfuYAN.github.io/MagicFight/ Dataset: https://huggingface.co/datasets/MingfuYAN/KungFu-Fiesta
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。