arXiv:2606.08402cs.CVcs.AI2026-06

用多智能体协作生成3D场景,从单张图还原空间结构与细节。

SceneConductor: 3D Scene Generation from a Single Image with Multi-Agent Orchestration

论文配图:SceneConductor: 3D Scene Generation from a Single Image with Multi-Agent Orchestration
图 1 · 摘自论文原文
  • 分三阶段生成:初始化、建环境、多智能体优化
  • 在ScanNet等数据集上几何精度和一致性显著提升
  • 适合需要高质量3D重建的工业与虚拟现实应用

从单张图像生成完整3D场景需从模糊视觉线索中推断全局一致的几何结构、物体关系与环境上下文。尽管联合布局与网格生成已有进展,现有方法多依赖整体或弱分解流程,同时耦合多种因素且依赖大量场景级标注,限制了在复杂真实场景中的泛化能力。本文提出多智能体编排框架,将单图3D场景生成分解为三个结构化阶段:场景初始化、环境构建与多智能体精修。初始化阶段提取图像中物体掩码,构建物体级3D表示并预测初始空间布局,形成粗粒度3D场景。环境构建阶段利用该初始化与点云地图几何信息,构建支撑面、房间边界、材质与光照等环境骨架。精修阶段由规划者智能体识别结构与视觉不一致,执行简单修正,并调度专业智能体进行局部复杂修订,再重新融入全局场景。为提供可靠结构初始化并减少对场景级标注的依赖,进一步引入基于稀疏几何先验(来自点图)的几何感知布局预测器。该预测器可从分割级数据训练,泛化能力强,适用于多样真实场景。在基准数据集上的大量实验表明,本方法在几何准确性、空间一致性与感知真实感方面均持续优于先前方法。

原文摘要 · Abstract (English)

Generating complete 3D scenes from a single image requires inferring globally consistent geometry, object relationships, and environmental context from inherently ambiguous visual evidence. Despite recent progress in joint layout-and-mesh generation, existing methods often rely on holistic or weakly decomposed pipelines that entangle many factors at once and demand extensive scene-level supervision, limiting their generalization to complex real-world environments. We propose a multi-agent orchestration framework that decomposes single-image 3D scene generation into three structured stages: scene initialization, environment construction, and multi-agent refinement. The initialization stage extracts image-derived object masks, builds object-level 3D representations, and predicts an initial spatial layout to form a coarse 3D scene. The environment-construction stage then leverages this initialization together with point-map geometry to build an environmental scaffold of supporting surfaces, room boundaries, materials, and illumination. Finally, in the refinement stage, a planner agent identifies structural and visual inconsistencies, applies simple corrections directly, and dispatches specialist agents for complex localized revisions that are reintegrated into the global scene. To provide reliable structural initialization while reducing reliance on scene-level annotations, we further introduce a geometry-aware layout predictor supervised by sparse geometric priors derived from point maps. Unlike fully supervised layout generators, the predictor can be trained from segmentation-level data and generalizes robustly to diverse real-world scenes. Extensive experiments on benchmark datasets show that our method consistently outperforms prior approaches in geometric accuracy, spatial consistency, and perceptual realism.

3D生成多智能体单图重建

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。