利用多视角信息提升镜面检测精度，改善含镜场景的3D重建质量。
MVMD: A Multi-View Approach for Enhanced Mirror Detection
- 通过跨视角与自注意力机制建模物体在不同视角间的关联。
- 相比单图方法，精度提升2.6%，交并比提高11.1%。
- 适合需要高精度3D重建的镜面密集场景应用。
在3D重建中，镜子会造成空间扭曲和碎片化，导致模型不准确。传统方法依赖单图检测，忽略了多视角提供的丰富信息。为此，我们提出MVMD（多视角镜面检测）方法，并构建首个面向多视角镜面检测的数据库。MVMD基于不同视角下物体与其镜像间的内在关联，利用交叉与自注意力机制学习这些关系。其包含三个模块：跨视角块追踪因视角变化引起的镜内物体位移；单视角块检测镜中反射物体；精炼块优化镜面边界并增强细节。实验表明，该方法相较单图检测技术，精度提升最高达2.6%，交并比（IoU）提升最高11.1%。这一显著改进使MVMD在镜面密集环境下的3D重建任务中表现尤为突出。
原文摘要 · Abstract (English)
In 3D reconstruction, mirrors introduce significant challenges by creating distorted and fragmented spaces, resulting in inaccurate and unreliable 3D models. As 3D reconstruction typically relies on multi-view images to capture different perspectives of a scene, detecting and labeling mirrors in multi-view images before reconstruction can effectively address this issue. However, existing methods focus solely on single-image detection, overlooking the rich information provided by multi-view setups. To overcome this limitation, we propose MVMD, a novel Multi-View Mirror Detection method, along with the first database specifically designed for mirror detection in multi-view scenes. The design of MVMD is grounded in the inherent associations between objects seen from different views and those reflected inside and outside of mirrors. These relationships are learned through cross- and self-attention mechanisms. MVMD consists of three key blocks: the Inter-Views Block tracks the shifts of objects within mirrors caused by changes in viewpoint; the Intra-View Block detects object reflections inside mirrors; and the Refinement Block sharpens mirror boundaries and enhances detected details. Experimental results show that our method improves accuracy by up to 2.6% and IoU by up to 11.1%, compared to single-image mirror detection techniques. This substantial improvement makes MVMD particularly effective for computer vision tasks, especially in enhancing the accuracy of 3D reconstruction in mirror-dense environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。