融合显式与隐式方法,提升3D重建的泛化性与速度。
H3R: Hybrid Multi-view Correspondence for Generalizable 3D Reconstruction
- 用体积隐变量+注意力特征聚合,兼顾几何精度与鲁棒性。
- 相比现有方法快2倍,多数据集上PSNR提升最高达1.06 dB。
- 适合需要高分辨率、多视角输入的3D重建场景。
尽管前向3D高斯泼溅取得进展,泛化性3D重建仍具挑战,尤其在多视角对应建模方面。现有方法存在根本权衡:显式方法精度高但难以处理模糊区域,隐式方法鲁棒性强但收敛慢。我们提出H3R,通过体素隐变量融合与基于注意力的特征聚合,集成两种范式。框架包含两个互补组件:利用对极约束保证几何一致性的高效隐体积,以及使用普吕克坐标自适应优化对应关系的相机感知Transformer。该方法显著提升泛化能力,收敛速度比现有方法快2倍。此外,我们发现空间对齐的基础模型(如SD-VAE)远优于语义对齐模型(如DINOv2),解决了语义表示与空间重建需求间的不匹配。本方法支持可变数量和高分辨率输入视图,并在跨数据集上表现稳健。大量实验表明,在RealEstate10K、ACID和DTU数据集上,分别实现0.59 dB、1.06 dB和0.22 dB的显著PSNR提升。代码已开源。
原文摘要 · Abstract (English)
Despite recent advances in feed-forward 3D Gaussian Splatting, generalizable 3D reconstruction remains challenging, particularly in multi-view correspondence modeling. Existing approaches face a fundamental trade-off: explicit methods achieve geometric precision but struggle with ambiguous regions, while implicit methods provide robustness but suffer from slow convergence. We present H3R, a hybrid framework that addresses this limitation by integrating volumetric latent fusion with attention-based feature aggregation. Our framework consists of two complementary components: an efficient latent volume that enforces geometric consistency through epipolar constraints, and a camera-aware Transformer that leverages Plücker coordinates for adaptive correspondence refinement. By integrating both paradigms, our approach enhances generalization while converging 2$\times$ faster than existing methods. Furthermore, we show that spatial-aligned foundation models (e.g., SD-VAE) substantially outperform semantic-aligned models (e.g., DINOv2), resolving the mismatch between semantic representations and spatial reconstruction requirements. Our method supports variable-number and high-resolution input views while demonstrating robust cross-dataset generalization. Extensive experiments show that our method achieves state-of-the-art performance across multiple benchmarks, with significant PSNR improvements of 0.59 dB, 1.06 dB, and 0.22 dB on the RealEstate10K, ACID, and DTU datasets, respectively. Code is available at https://github.com/JiaHeng-DLUT/H3R.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。