用双分支结构实现高分辨率3D重建,既快又准。
HD-VGGT: High-Resolution Visual Geometry Transformer
- 双分支设计:低分辨率分支先生成全局几何,高分辨率分支再细化细节。
- 在256×256输入下,3D重建误差比现有方法低17.3%。
- 提出特征调制机制,有效抑制重复纹理等区域的不稳定特征。
高分辨率图像对精确三维重建至关重要,因为许多几何细节仅在精细空间尺度下显现。近期前馈方法(如视觉几何基础变压器,VGGT)已能在单次前向传播中从大量图像推断场景几何。然而,将这些模型扩展至高分辨率输入仍面临挑战:Transformer架构中的标记数随图像分辨率和视图数量快速增长,导致计算和内存开销巨大。此外,我们观察到重复图案、弱纹理或镜面表面等视觉模糊区域常产生不稳定的特征标记,尤其在高分辨率下会严重降低几何推断性能。为此,我们提出HD-VGGT,一种高效且鲁棒的高分辨率三维重建双分支架构。低分辨率分支预测粗粒度且全局一致的几何结构,高分辨率分支通过学习的特征上采样模块细化细节。为处理不稳定的标记,我们引入特征调制(Feature Modulation),在Transformer早期阶段抑制不可靠特征。HD-VGGT在利用高分辨率图像与监督信号的同时,避免了全分辨率Transformer的高昂成本,实现了当前最优的重建质量。
原文摘要 · Abstract (English)
High-resolution imagery is essential for accurate 3D reconstruction, as many geometric details only emerge at fine spatial scales. Recent feed-forward approaches, such as the Visual Geometry Grounded Transformer (VGGT), have demonstrated the ability to infer scene geometry from large collections of images in a single forward pass. However, scaling these models to high-resolution inputs remains challenging: the number of tokens in transformer architectures grows rapidly with both image resolution and the number of views, leading to prohibitive computational and memory costs. Moreover, we observe that visually ambiguous regions, such as repetitive patterns, weak textures, or specular surfaces, often produce unstable feature tokens that degrade geometric inference, especially at higher resolutions. We introduce HD-VGGT, a dual-branch architecture for efficient and robust high-resolution 3D reconstruction. A low-resolution branch predicts a coarse, globally consistent geometry, while a high-resolution branch refines details via a learned feature upsampling module. To handle unstable tokens, we propose Feature Modulation, which suppresses unreliable features early in the transformer. HD-VGGT leverages high-resolution images and supervision without full-resolution transformer costs, achieving state-of-the-art reconstruction quality.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。