arXiv:2502.20861cs.CV2025-02CVPR被引 1

从单图重建3D形状,主动挖掘有效语义线索提升精度。

MESC-3D:Mining Effective Semantic Cues for 3D Reconstruction from a Single Image

  • 设计语义挖掘模块,让点云自主选择图像中的关键语义信息。
  • 引入3D语义先验学习模块,显著提升对遮挡等困难场景的重建能力。
  • 在未见类别上表现优异,具备强泛化能力,适合复杂场景重建任务。

从单张图像重建3D形状在计算机视觉中具有重要意义。尽管已有方法取得显著进展,但多数仅简单将图像语义与3D点云拼接,未深入挖掘融合后的语义特征,导致语义混淆严重,影响重建效果。本文提出MESC-3D,一种新型单图像3D重建方法,可主动挖掘被纠缠的语义线索。我们设计了有效语义挖掘模块,建立点云与图像语义属性间的关联,使点云能自主选择所需信息。此外,为弥补单图语义信息不足(如遮挡),受人类基于日常经验构建3D认知的启发,引入3D语义先验学习模块,融合空间结构的语义理解,使模型更准确、真实地重构3D物体,贴近人类对复杂3D环境的感知。大量实验表明,该方法在重建质量与鲁棒性上显著优于现有方法。进一步实验验证其强大的泛化能力,在未见类别上表现出色,零样本性能优异。代码已公开于https://github.com/QINGQINGLE/MESC-3D。

原文摘要 · Abstract (English)

Reconstructing 3D shapes from a single image plays an important role in computer vision. Many methods have been proposed and achieve impressive performance. However, existing methods mainly focus on extracting semantic information from images and then simply concatenating it with 3D point clouds without further exploring the concatenated semantics. As a result, these entangled semantic features significantly hinder the reconstruction performance. In this paper, we propose a novel single-image 3D reconstruction method called Mining Effective Semantic Cues for 3D Reconstruction from a Single Image (MESC-3D), which can actively mine effective semantic cues from entangled features. Specifically, we design an Effective Semantic Mining Module to establish connections between point clouds and image semantic attributes, enabling the point clouds to autonomously select the necessary information. Furthermore, to address the potential insufficiencies in semantic information from a single image, such as occlusions, inspired by the human ability to represent 3D objects using prior knowledge drawn from daily experiences, we introduce a 3D Semantic Prior Learning Module. This module incorporates semantic understanding of spatial structures, enabling the model to interpret and reconstruct 3D objects with greater accuracy and realism, closely mirroring human perception of complex 3D environments. Extensive evaluations show that our method achieves significant improvements in reconstruction quality and robustness compared to prior works. Additionally, further experiments validate the strong generalization capabilities and excels in zero-shot preformance on unseen classes. Code is available at https://github.com/QINGQINGLE/MESC-3D.

3D重建单图生成语义挖掘先验知识

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。