arXiv:2601.10909cs.CV2026-01被引 8

首次实现身体各部位的精细动作生成与组合,支持逐帧控制。

FrankenMotion: Part-level Human Motion Generation and Composition

  • 用大模型构建细粒度部位动作标注数据集,支持时间敏感的原子级描述。
  • 提出扩散模型框架,每个身体部位独立接收时序文本引导,生成更可控动作。
  • 可合成训练中未见的动作组合,适合需要精细动作控制的研究者使用。

近年来,基于文本提示的人体动作生成取得了显著进展。然而,现有方法主要依赖序列级或动作级描述,因缺乏细粒度的身体部位级动作标注,难以对个体部位进行精确控制。本文构建了一个高质量的运动数据集,包含原子级、时间感知的部位级文本标注,利用大语言模型(LLMs)的推理能力实现标注。与以往在固定时间段同步标注部位或仅使用全局序列标签的数据集不同,本数据集捕捉了异步且语义分明的部位运动,在细粒度时间分辨率下呈现。基于该数据集,我们提出一种基于扩散模型的部位感知生成框架——FrankenMotion,其中每个身体部位由其自身的时序结构化文本提示引导。据我们所知,这是首个提供原子级、时间感知的部位级动作标注,并具备空间(部位)和时间(原子动作)双重控制能力的模型。实验表明,FrankenMotion在所有适配并重新训练的基线模型中表现更优,且能合成训练中未见过的动作组合。代码与数据集将在发表后公开。

原文摘要 · Abstract (English)

Human motion generation from text prompts has made remarkable progress in recent years. However, existing methods primarily rely on either sequence-level or action-level descriptions due to the absence of fine-grained, part-level motion annotations. This limits their controllability over individual body parts. In this work, we construct a high-quality motion dataset with atomic, temporally-aware part-level text annotations, leveraging the reasoning capabilities of large language models (LLMs). Unlike prior datasets that either provide synchronized part captions with fixed time segments or rely solely on global sequence labels, our dataset captures asynchronous and semantically distinct part movements at fine temporal resolution. Based on this dataset, we introduce a diffusion-based part-aware motion generation framework, namely FrankenMotion, where each body part is guided by its own temporally-structured textual prompt. This is, to our knowledge, the first work to provide atomic, temporally-aware part-level motion annotations and have a model that allows motion generation with both spatial (body part) and temporal (atomic action) control. Experiments demonstrate that FrankenMotion outperforms all previous baseline models adapted and retrained for our setting, and our model can compose motions unseen during training. Our code and dataset will be publicly available upon publication.

动作生成扩散模型部位控制文本生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。