用剧情驱动生成3D场景中多类型真人动作,提升动漫游戏制作效率。
Sitcom-Crafter: A Plot-Driven Human Motion Generation System in 3D Scenes
- 基于8个模块的系统,以剧情为引导生成多样化动作。
- 通过隐式SDF点减少人与场景碰撞,无需额外数据收集。
- 适合动漫、游戏设计师快速生成真实感动作序列。
当前人体动作生成研究多聚焦于特定动作类型,如人-场景交互、行走或人-人互动,但缺乏能统一生成多种动作组合的系统。为此,我们提出Sitcom-Crafter,一个可扩展的3D空间人体动作生成系统,可通过丰富剧情上下文提升动漫与游戏设计的工作流效率。系统包含八个模块,其中三个用于动作生成,其余五个为增强模块,确保动作序列的一致融合与系统功能完整性。核心生成模块包含新颖的3D场景感知人-人交互模块,通过在运动空间周围合成隐式3D有符号距离函数(SDF)点,有效避免人-场景碰撞,且无需额外数据采集成本。此外,行走与人-场景交互模块采用现有方法以增强生成能力。增强模块包括剧情理解用于指令生成、动作同步实现不同动作类型无缝衔接、手部姿态检索提升动作真实感、动作碰撞修正防止人物间碰撞,以及3D重定向保障视觉保真度。实验验证了系统生成高质量、多样化且物理合理的动作的能力,展现出其在创意工作流中的应用潜力。
原文摘要 · Abstract (English)
Recent advancements in human motion synthesis have focused on specific types of motions, such as human-scene interaction, locomotion or human-human interaction, however, there is a lack of a unified system capable of generating a diverse combination of motion types. In response, we introduce Sitcom-Crafter, a comprehensive and extendable system for human motion generation in 3D space, which can be guided by extensive plot contexts to enhance workflow efficiency for anime and game designers. The system is comprised of eight modules, three of which are dedicated to motion generation, while the remaining five are augmentation modules that ensure consistent fusion of motion sequences and system functionality. Central to the generation modules is our novel 3D scene-aware human-human interaction module, which addresses collision issues by synthesizing implicit 3D Signed Distance Function (SDF) points around motion spaces, thereby minimizing human-scene collisions without additional data collection costs. Complementing this, our locomotion and human-scene interaction modules leverage existing methods to enrich the system's motion generation capabilities. Augmentation modules encompass plot comprehension for command generation, motion synchronization for seamless integration of different motion types, hand pose retrieval to enhance motion realism, motion collision revision to prevent human collisions, and 3D retargeting to ensure visual fidelity. Experimental evaluations validate the system's ability to generate high-quality, diverse, and physically realistic motions, underscoring its potential for advancing creative workflows. Project page: https://windvchen.github.io/Sitcom-Crafter.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。