arXiv:2602.20476cs.CV2026-02被引 3

用2D场景图生成逼真3D人体动作,效率更高且更物理合理。

SceMoS: Scene-Aware 3D Human Motion Synthesis by Planning with Geometry-Grounded Tokens

  • 用鸟瞰图和高度图分离全局规划与局部执行
  • 在TRUMANS上达到最优动作真实度与接触准确率
  • 仅需一半参数量,适合高效场景交互生成

文本驱动的3D人体动作合成需同时理解语义意图(如“走到沙发”)和物理可行性(如避障)。现有方法依赖高计算成本的3D场景数据(如点云或体素网格),并同时学习高层规划与底层接触推理。本文提出SceMoS,首次证明结构化的2D场景表示可替代完整3D监督,实现物理合理的动作生成。SceMoS通过轻量级2D线索解耦全局规划与局部执行:(1) 基于文本条件的自回归全局规划器,使用从场景高角渲染的鸟瞰图(BEV)作为输入,经由DINOv2编码;(2) 通过条件VQ-VAE训练的几何接地动作分词器,利用2D局部高度图将表面物理嵌入离散词汇。该2D分解实现效率与保真度的平衡:BEV语义捕捉空间布局与可操作性,用于全局推理;局部高度图强制细粒度物理一致性,无需全3D体素推理。SceMoS在TRUMANS基准上达到当前最优的动作真实度与接触准确率,且场景编码可训练参数减少超过50%,表明2D场景线索可有效支撑3D人体-场景交互。

原文摘要 · Abstract (English)

Synthesizing text-driven 3D human motion within realistic scenes requires learning both semantic intent ("walk to the couch") and physical feasibility (e.g., avoiding collisions). Current methods use generative frameworks that simultaneously learn high-level planning and low-level contact reasoning, and rely on computationally expensive 3D scene data such as point clouds or voxel occupancy grids. We propose SceMoS, a scene-aware motion synthesis framework that shows that structured 2D scene representations can serve as a powerful alternative to full 3D supervision in physically grounded motion synthesis. SceMoS disentangles global planning from local execution using lightweight 2D cues and relying on (1) a text-conditioned autoregressive global motion planner that operates on a bird's-eye-view (BEV) image rendered from an elevated corner of the scene, encoded with DINOv2 features, as the scene representation, and (2) a geometry-grounded motion tokenizer trained via a conditional VQ-VAE, that uses 2D local scene heightmap, thus embedding surface physics directly into a discrete vocabulary. This 2D factorization reaches an efficiency-fidelity trade-off: BEV semantics capture spatial layout and affordance for global reasoning, while local heightmaps enforce fine-grained physical adherence without full 3D volumetric reasoning. SceMoS achieves state-of-the-art motion realism and contact accuracy on the TRUMANS benchmark, reducing the number of trainable parameters for scene encoding by over 50%, showing that 2D scene cues can effectively ground 3D human-scene interaction.

动作生成2D场景物理合理性高效建模

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。