arXiv:2603.11633cs.CV2026-03被引 11

多视角融合生成3D场景,让物体摆放更真实合理。

MV-SAM3D: Adaptive Multi-View Fusion for Layout-Aware 3D Generation

  • 用3D隐空间的多扩散机制融合多视角信息
  • 通过自适应加权提升观测可靠性,避免物体穿插
  • 无需训练,适合需要真实布局的3D生成场景

统一的3D生成模型在单图生成高质量3D资产方面取得显著进展。例如,布局感知方法SAM3D可重建多个物体并保持其空间排列,为场景级3D生成打开新路径。然而,现有方法仅支持单视角输入,无法利用多视角互补信息,且独立估计的物体姿态常导致物理上不合理的布局,如物体穿插和漂浮。我们提出MV-SAM3D,一个无需训练的框架,扩展了布局感知3D生成的多视角一致性与物理合理性。我们将多视角融合建模为3D隐空间中的多扩散过程,并提出两种自适应加权策略——注意力熵加权和可见性加权,实现置信度感知融合,确保每个视点按其观测可靠性贡献。针对多物体组合,引入物理感知优化,在生成过程中及之后注入碰撞与接触约束,获得物理上合理的物体布局。在标准基准和真实世界多物体场景上的实验表明,该方法在重建保真度和布局合理性上均有显著提升,且无需额外训练。代码已开源。

原文摘要 · Abstract (English)

Recent unified 3D generation models have made remarkable progress in producing high-quality 3D assets from a single image. Notably, layout-aware approaches such as SAM3D can reconstruct multiple objects while preserving their spatial arrangement, opening the door to practical scene-level 3D generation. However, current methods are limited to single-view input and cannot leverage complementary multi-view observations, while independently estimated object poses often lead to physically implausible layouts such as interpenetration and floating artifacts. We present MV-SAM3D, a training-free framework that extends layout-aware 3D generation with multi-view consistency and physical plausibility. We formulate multi-view fusion as a Multi-Diffusion process in 3D latent space and propose two adaptive weighting strategies -- attention-entropy weighting and visibility weighting -- that enable confidence-aware fusion, ensuring each viewpoint contributes according to its local observation reliability. For multi-object composition, we introduce physics-aware optimization that injects collision and contact constraints both during and after generation, yielding physically plausible object arrangements. Experiments on standard benchmarks and real-world multi-object scenes demonstrate significant improvements in reconstruction fidelity and layout plausibility, all without any additional training. Code is available at https://github.com/devinli123/MV-SAM3D.

3D生成多视角融合物理合理性无需训练

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。