统一人体动作生成与编辑,用条件驱动实现灵活控制。
MotionLab: Unified Human Motion Generation and Editing via the Motion-Condition-Motion Paradigm
- 提出'动作用条件生成目标动作'的统一框架
- 支持多任务无缝切换,生成与编辑性能优异
- 适合需要精细动作控制的动画与虚拟人应用
人体动作生成与编辑是计算机视觉的关键任务。现有方法多针对特定任务设计,缺乏统一性与可编辑性。为此,本文提出新范式——‘运动-条件-运动’,将各类任务统一为源动作、条件与目标动作三要素。基于此,构建统一框架 MotionLab,利用修正流学习从源动作到目标动作的映射。核心组件包括:1)MotionFlow Transformer,实现无任务专用模块的条件生成与编辑;2)对齐旋转位置编码,保证源动作与目标动作的时间同步;3)任务指定指令调制;4)运动课程学习,促进多任务学习与知识共享。实验表明,MotionLab 在多个基准上展现出优异的泛化能力与推理效率。代码与视频结果见:https://diouo.github.io/motionlab.github.io/
原文摘要 · Abstract (English)
Human motion generation and editing are key components of computer vision. However, current approaches in this field tend to offer isolated solutions tailored to specific tasks, which can be inefficient and impractical for real-world applications. While some efforts have aimed to unify motion-related tasks, these methods simply use different modalities as conditions to guide motion generation. Consequently, they lack editing capabilities, fine-grained control, and fail to facilitate knowledge sharing across tasks. To address these limitations and provide a versatile, unified framework capable of handling both human motion generation and editing, we introduce a novel paradigm: \textbf{Motion-Condition-Motion}, which enables the unified formulation of diverse tasks with three concepts: source motion, condition, and target motion. Based on this paradigm, we propose a unified framework, \textbf{MotionLab}, which incorporates rectified flows to learn the mapping from source motion to target motion, guided by the specified conditions. In MotionLab, we introduce the 1) MotionFlow Transformer to enhance conditional generation and editing without task-specific modules; 2) Aligned Rotational Position Encoding to guarantee the time synchronization between source motion and target motion; 3) Task Specified Instruction Modulation; and 4) Motion Curriculum Learning for effective multi-task learning and knowledge sharing across tasks. Notably, our MotionLab demonstrates promising generalization capabilities and inference efficiency across multiple benchmarks for human motion. Our code and additional video results are available at: https://diouo.github.io/motionlab.github.io/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。