arXiv:2606.28060cs.CV2026-06

ReScene通过多视角几何融合,实现物理合理且结构清晰的室内场景重建。

ReScene: Structured Indoor Scene Reconstruction from Multi-View Captures

论文配图:ReScene: Structured Indoor Scene Reconstruction from Multi-View Captures
图 1 · 摘自论文原文
  • 基于语义一致性和3D覆盖率选择重建视角,避免误判物体覆盖范围。
  • 融合视觉-语言模型的关系预测与几何先验,构建置信加权场景图,提升物理一致性。
  • 在扫描数据上性能超越现有方法17%(切比雪夫距离)和26%(感知质量),速度更快。

从多视角图像重建可用于仿真的3D场景是具身人工智能的关键瓶颈,因下游任务需要对象级结构、明确的物体间关系以及物理合理性。现有方法或依赖专用捕获设备,或在单视角重建中存在偏差,或生成几何合理但物理不一致的布局。我们发现核心问题并非单物体重建,而是跨视角关系融合与物理合理的场景组装。为此,我们提出ReScene框架,将多视角几何作为贯穿全流程的统一先验。该方法包含两个主要组件:HierView根据语义一致性和3D覆盖完整性优先选择重建视角,取代混淆图像占据与物体覆盖的‘最大掩码’启发式;Relation-Aware Assembly将视觉-语言模型的多帧关系预测与几何及房间外壳先验结合,形成置信度加权的场景图,实现物理一致的场景组装。ReScene在一组ScanNet场景上达到新基准,在几何、渲染和感知质量方面超越最强基线,切比雪夫距离降低17%,LPIPS下降26%,运行速度比先前多视图方法快达10倍。基于重建场景,我们还构建了一个具身视觉问答数据集,微调后的Qwen-VL在多个空间推理任务上接近强闭源模型表现。

原文摘要 · Abstract (English)

Constructing simulation-ready 3D scenes from multi-view captures is a key bottleneck for Embodied Artificial Intelligence, as downstream tasks require object-level structure, explicit inter-object relations, and physical plausibility. Existing approaches either rely on specialized capture hardware, suffer from single-view bias in object reconstruction, or yield layouts that are geometrically reasonable but physically inconsistent. We identify that the problem is not single-object reconstruction but cross-view relation fusion and physically plausible scene assembly. To address this challenge, we present ReScene, a framework that threads multi-view geometry throughout the pipeline as a unifying prior. Our method consists of two main components: HierView prioritizes reconstruction views based on semantic consistency and 3D coverage completeness, replacing the largest-mask heuristic that conflates image occupancy with object coverage; and Relation-Aware Assembly fuses multi-frame relation predictions from a vision-language model with geometric and room-shell priors into a confidence-weighted scene graph, enabling physically consistent scene assembly. ReScene sets a new state of the art across geometry, rendering, and perceptual quality on a set of ScanNet scenes, achieving a 17% reduction in Chamfer Distance and 26% in LPIPS over the strongest prior baseline, while running up to 10x faster than prior multi-view methods. Based on the reconstructed scenes, we also generate an embodied visual question answering dataset, on which fine-tuned Qwen-VL approaches the performance of strong closed-source models on several spatial reasoning tasks.

场景重建多视角物理一致性具身AI

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。