arXiv:2603.08590cs.CV2026-03被引 1

用关节级潜在变量分解,实现可控的实时人体动作生成。

PRISM: Streaming Human Motion Generation with Per-Joint Latent Decomposition

  • 将动作分解为关节级潜在变量,显式建模身体结构。
  • 1.4B参数模型在公开数据上超越所有学术基线,接近私有数据水平。
  • 支持逐帧续动生成与长序列拼接,适合交互式应用。

文本到动作生成已随大规模数据和更强生成器发展,但多数模型仍依赖整体帧或片段级潜在表示,纠缠了轨迹、朝向与关节运动。这种纠缠模糊了身体拓扑,迫使生成器隐式恢复运动学结构。我们提出 ame,一种基于SMPL的动作生成框架,将动作分解为连续的运动学单元潜在变量。一个因果运动变分自编码器(Motion VAE)将动作映射到时间-运动学单元潜在流形,一个运动学单元流变换器在该结构化空间中进行文本条件的流匹配。由于每个潜在坐标始终对应物理身体单元, ame 可使用运动学树旋转变换位置编码和运动学自适应流调度。我们还通过每标记时间步的干净上下文/噪声目标掩码训练生成器,实现帧条件续写与自回归段落链式生成。实验验证:运动学单元VAE在几何、旋转和特征误差上均优于现有动作分词器,表明潜在空间保留了骨架结构而非仅压缩帧。使用仅1.4B参数的生成器在公开学术数据上训练, ame 超越所有评估的学术数据基线,并保持与使用更大非公开数据集系统相当的竞争力。无需任务特定微调,同一架构也提升前缀条件生成、BABEL序列滚动与叙事动作组合效果。结果表明,运动学单元潜在分解为可控的SMPL动作合成提供了有效基础。代码将在 https://github.com/ZeyuLing/PRISM 发布。

原文摘要 · Abstract (English)

Text-to-motion generation has advanced with larger corpora and stronger generators, yet many models still rely on holistic frame- or clip-level latents that entangle trajectory, orientation, and articulation. This entanglement obscures body topology and forces the generator to recover kinematic structure implicitly. We present \name, a SMPL motion generation framework that factorizes motion into continuous kinematic-unit latents. A causal Motion VAE maps motion to a time-by-kinematic-unit latent manifold, and a Kinematic-Unit Flow Transformer performs text-conditioned flow matching in this structured space. Because each latent coordinate remains tied to a physical body unit, \name can use kinematic-tree rotary position encoding and kinematic-adaptive flow scheduling. We further train the generator with per-token timesteps over clean-context/noisy-target masks, enabling frame-conditioned continuation and autoregressive segment chaining within one model. Experiments first validate the representation: the kinematic-unit VAE achieves lower geometry, rotation, and feature errors than existing motion tokenizers, showing that the latent space preserves articulated structure rather than merely compressing frames. With a 1.4B-parameter generator trained only on publicly available academic motion--text data, \name outperforms all evaluated academic-data text-to-motion baselines and remains competitive with systems trained on much larger non-public motion corpora. Without task-specific retraining, the same formulation also improves prefix-conditioned generation, BABEL sequential rollout, and narrative motion composition. These results indicate that kinematic-unit latent factorization provides an effective generation substrate for controllable SMPL motion synthesis. Code will be released at https://github.com/ZeyuLing/PRISM.

动作生成潜变量分解实时生成SMPL

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。