将3D场景中的运动轨迹智能迁移至语义相似的另一场景,避免碰撞和扭曲。
Analogical Trajectory Transfer

- 按物体中心分块,用3D基础模型特征预测跨场景映射
- 0.6秒完成转移,优于基于LLM/VLM的基线方法
- 适用于虚拟共存、人到机器人的动作迁移等场景
我们研究类比轨迹迁移,目标是将一个3D环境中的运动轨迹转移到另一个语义相似的环境中。这种能力使机器能进行类比空间推理,应用于增强现实/虚拟现实共现、内容创作和机器人学。然而,即使语义相似的场景在物体布局、尺度和结构上仍可能差异显著,直接匹配语义会导致碰撞或几何失真。同时,确定每个轨迹点应映射的位置搜索空间巨大,需保持语义一致性和功能完整性,避免轨迹撕裂或碰撞。我们的关键洞察是将问题分解为独立的空间子问题,并合并解以生成语义一致且空间连贯的转移结果。具体而言,将场景划分为以物体为中心的簇,利用编码了物体与开放空间布局上下文信息的3D基础模型特征,通过层级平滑映射预测实现跨场景映射。随后组合各簇映射生成初始转移,并优化以消除碰撞与失真,最终获得空间连贯的轨迹。本方法无需训练,运行时间约0.6秒,优于基于LLM、VLM及场景图匹配的基线。我们进一步展示了在虚拟共现、多轨迹迁移、相机轨迹迁移以及人到机器人动作迁移中的应用,表明其在AR/VR与机器人领域的广泛适用性。
原文摘要 · Abstract (English)
We study analogical trajectory transfer, where the goal is to translate motion trajectories in one 3D environment to a semantically analogous location in another. Such a capacity would enable machines to perform analogical spatial reasoning, with applications in AR/VR co-presence, content creation, and robotics. However, even semantically similar scenes can still differ substantially in object placement, scale, and layout, so naively matching semantics leads to collisions or geometric distortions. Furthermore, finding where each trajectory point should transfer to has a large search space, as the mapping must preserve semantics and functionality without tearing the trajectory apart or causing collisions. Our key insight is to decompose the problem into spatially segregated subproblems and merge their solutions to produce semantically consistent and spatially coherent transfers. Specifically, we partition scenes into object-centric clusters and estimate cross-scene mappings via hierarchical smooth map prediction, using 3D foundation model features that encode contextual information from object and open-space arrangements. We then combinatorially assemble the per-cluster maps into an initial transfer and refine the result to remove collisions and distortions, yielding a spatially coherent trajectory. Our method does not require training, attains a fast runtime around 0.6 seconds, and outperforms baselines based on LLMs, VLMs, and scene graph matching. We further showcase applications in virtual co-presence, multi-trajectory transfer, camera transfer, and human-to-robot motion transfer, which indicates the broad applicability of our work to AR/VR and robotics.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。