arXiv:2608.30821cs.CVcs.AI2026-08

让机器人仿真环境更真实:从混乱实拍中重建可编辑的3D场景。

Lucida: Parse, Generate, and Place for Composable Real-to-Sim Scene Modeling

论文配图:Lucida: Parse, Generate, and Place for Composable Real-to-Sim Scene Modeling
图 1 · 摘自论文原文
  • 分三步重构场景:解析视频成图谱,根据多视角证据生成物体资产,用AI自动放置到位。
  • 在多个数据集上性能大幅提升,最高提升69%的检测准确率,物体姿态误差降低近30%。
  • 适合做机器人仿真、具身智能研究者,尤其关注真实场景重建与可控编辑的团队。

可组合场景建模旨在将真实室内场景还原为完整、可编辑的物体资产并按观察位置排列,为机器人仿真和具身AI提供可操作的真实环境副本。现有方法分为三步:解析观测结果、生成物体资产、重新放置。但每一步都依赖于难以获取的高质量输入——精确的实例几何、无遮挡视图及匹配观测的资产。本文提出Lucida,保持三步流程但重新分配需求,使每一步仅使用真实采集中可靠提供的信息,精度在流程末端达成而非初始即要求。Lucida将视频解析为包含多视角证据的场景图,基于证据生成完整物体资产,并通过GizmoAct(一种视觉语言模型策略)实现闭环放置,将放置视为多轮图形界面交互,自主判断对齐完成。在场景级3D物体检测、物体位姿估计和场景重建任务中,Lucida在R2S-Scene上相比Boxer的mAP提升69%,在CA-1M上[email protected]从57.8%提高至83.4%,场景F-Score从SAM3D的0.794增至0.924。

原文摘要 · Abstract (English)

Composable scene modeling aims to recover a real indoor scene as complete, editable object assets arranged as observed, giving robot simulation and embodied AI a simulation-ready replica of the real environment whose objects can be manipulated individually. Existing pipelines decompose the task into three steps---parse the observations into instances, generate an asset for each, and place each asset back---but every step presumes an input that a cluttered capture rarely provides: accurate instance geometry, unoccluded views, and assets that accurately match the observations. We propose Lucida, which keeps this order but redistributes the requirements, so each step consumes only what a real capture reliably provides and precision is reached at the end of the pipeline rather than demanded at its start. Lucida parses the video into a scene graph whose nodes carry per-instance multi-view evidence, generates a complete asset for each instance from its evidence, and places assets with GizmoAct, a VLM policy that casts placement as multi-turn GUI interaction, manipulating the object's gizmo in a closed loop and deciding itself when alignment is reached. Across scene-level 3D object detection, object pose estimation, and scene reconstruction, Lucida improves mAP over Boxer by 69% on R2S-Scene, raises [email protected] from 57.8% to 83.4% on CA-1M, and increases scene F-Score from 0.794 for SAM3D to 0.924.

场景建模机器人仿真视觉语言模型3D重建

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。