通过几何感知去噪提升多视角3D重建在真实退化场景下的鲁棒性。
Geometry-Aware Representation Denoising for Robust Multi-view 3D Reconstruction
- 在前馈3D重建模型的特征空间中进行扩散去噪，利用几何感知特征恢复结构。
- 在DA3基准上实现更高精度的3D几何重建与高质量图像复原。
- 适合需要真实场景鲁棒性的3D重建应用，如自动驾驶、数字孪生。
多视角3D重建因前馈式3D重建模型的发展取得了显著进展。然而，这些模型通常在理想无退化的成像条件下训练和评估，而真实观测常包含显著差异的退化因素。因此，在退化条件下提升多视角3D重建的鲁棒性仍是重要挑战。本文提出几何感知表示去噪（GARD）框架，该框架在前馈3D重建模型的特征空间中直接执行基于扩散的多视角修复。这一设计利用了3D重建器的几何感知特征表示，有效恢复精确的场景几何结构。此外，通过引入额外的RGB图像解码器，优化后的表示还可用于恢复高质量的RGB图像，从而实现3D场景几何与高质量影像的同时恢复。在Depth Anything 3（DA3）基准上的全面实验验证了所提GARD框架的有效性。
原文摘要 · Abstract (English)
Multi-view 3D reconstruction has achieved remarkable progress with the advent of feed-forward 3D reconstruction models. However, these models are typically trained and evaluated under ideal, degradation-free imaging conditions, whereas real-world observations often contain degradations that differ significantly from such settings. Improving robustness for multi-view 3D reconstruction under degraded conditions therefore remains an important challenge. We present Geometry-Aware Representation Denoising (GARD), a novel framework that performs diffusion-based multi-view restoration directly in the feature space of a feed-forward 3D reconstruction model. This design exploits the geometry-aware feature representations of the 3D reconstructor to effectively recover accurate scene geometry. Furthermore, by employing an additional RGB image decoder, the refined representations can also be used to restore high-quality RGB images, thereby enabling the simultaneous recovery of 3D scene geometry and high-quality imagery. Comprehensive experiments on the Depth Anything 3 (DA3) benchmark demonstrate the effectiveness of the proposed GARD framework.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。