arXiv:2411.17763cs.CV2024-11CVPR被引 10

从单图检测对称性,提升3D生成质量

Symmetry Strikes Back: From Single-Image Symmetry Detection to 3D Generation

  • 用Transformer架构实现零样本对称性检测
  • 在多种数据集上达到新最佳性能
  • 适合需要高质量3D重建的研究者

对称性是视觉世界中普遍且基础的属性,对感知和结构理解至关重要。本文研究从单张RGB图像中检测3D反射对称性,并揭示其在单图3D生成中的显著优势。我们提出Reflect3D,一种可扩展的零样本对称性检测器,能在多样且真实场景中稳健泛化。受基础模型成功启发,方法采用基于Transformer的架构,并利用多视角扩散模型的生成先验来解决单视角对称性检测的固有歧义问题。在多个数据源上的广泛评估表明,Reflect3D在单图对称性检测任务上建立了新基准。此外,通过引入对称性感知优化流程,将检测到的对称性融入单图3D生成管道,显著提升了重建3D几何与纹理的结构准确性、整体一致性和视觉保真度,推动了3D内容创作能力的发展。

原文摘要 · Abstract (English)

Symmetry is a ubiquitous and fundamental property in the visual world, serving as a critical cue for perception and structure interpretation. This paper investigates the detection of 3D reflection symmetry from a single RGB image, and reveals its significant benefit on single-image 3D generation. We introduce Reflect3D, a scalable, zero-shot symmetry detector capable of robust generalization to diverse and real-world scenarios. Inspired by the success of foundation models, our method scales up symmetry detection with a transformer-based architecture. We also leverage generative priors from multi-view diffusion models to address the inherent ambiguity in single-view symmetry detection. Extensive evaluations on various data sources demonstrate that Reflect3D establishes a new state-of-the-art in single-image symmetry detection. Furthermore, we show the practical benefit of incorporating detected symmetry into single-image 3D generation pipelines through a symmetry-aware optimization process. The integration of symmetry significantly enhances the structural accuracy, cohesiveness, and visual fidelity of the reconstructed 3D geometry and textures, advancing the capabilities of 3D content creation.

3D生成对称性检测扩散模型单图重建

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。