仅用一张图生成任意物体的多样3D动作，支持语言控制与插值。
DIMO: Diverse 3D Motion Generation for Arbitrary Objects
- 利用视频模型先验提取通用运动模式，构建紧凑的神经关键点轨迹表示。
- 在单一前向传播中实现多样3D动作生成，支持动作插值与语言引导生成。
- 基于共享低维隐空间，可快速采样新动作，适用于交互式创作场景。
我们提出DIMO，一种从单张图像生成任意物体多样3D动作的生成方法。核心思路是利用训练良好的视频模型中的丰富先验，提取通用运动模式，并将其嵌入共享的低维隐空间。具体而言，首先生成同一物体的多段不同动作视频，将每段动作编码为隐向量，并训练一个共享动作解码器，学习由结构化、紧凑的神经关键点轨迹所表征的动作分布。随后，通过这些关键点驱动标准3D高斯并融合以建模几何与外观。推理时，基于已学习的隐空间，可在单次前向传播中即时采样多样3D动作，支持3D动作插值与语言引导动作生成等应用。项目主页见 https://linzhanm.github.io/dimo。
原文摘要 · Abstract (English)
We present DIMO, a generative approach capable of generating diverse 3D motions for arbitrary objects from a single image. The core idea of our work is to leverage the rich priors in well-trained video models to extract the common motion patterns and then embed them into a shared low-dimensional latent space. Specifically, we first generate multiple videos of the same object with diverse motions. We then embed each motion into a latent vector and train a shared motion decoder to learn the distribution of motions represented by a structured and compact motion representation, i.e., neural key point trajectories. The canonical 3D Gaussians are then driven by these key points and fused to model the geometry and appearance. During inference time with learned latent space, we can instantly sample diverse 3D motions in a single-forward pass and support several interesting applications including 3D motion interpolation and language-guided motion generation. Our project page is available at https://linzhanm.github.io/dimo.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。