用多块分治法实现单目高分辨率几何估计,兼顾细节与全局一致性。
Any Resolution Any Geometry: From Multi-View To Multi-Patch
- 将图像切块并注入粗略深度和法向先验,统一用变换器联合推理
- 在UnrealStereo4K上精度提升:AbsRel降至0.0291,角误差降为18.51度
- 支持零样本迁移与超高清扩展,适合需要高保真三维重建的场景
联合估计表面法向与深度对全面理解三维场景至关重要,但高分辨率预测仍面临局部细节与全局一致性之间的权衡。为此,我们提出超分辨率几何变换器(URGT),将视觉几何基础变换器(VGGT)改造为统一的多块变换器,用于单目高分辨率深度-法向估计。单张高分辨率图像被分割为多个块,结合预训练模型提供的粗略深度和法向先验,在一次前向传播中联合处理以生成优化的几何输出。通过跨块注意力机制强化全局一致性,实现长距离几何推理及块间信息无缝传递。为进一步提升空间鲁棒性,引入GridMix块采样策略,在训练中概率性采样网格配置,增强块间一致性与泛化能力。该方法在UnrealStereo4K数据集上取得当前最优表现,深度估计的AbsRel从0.0582降至0.0291,RMSE由2.17降至1.31,平均角误差从23.36度降至18.51度,同时生成更清晰稳定的几何结构。所提出的多块框架展现出强零样本与跨域泛化能力,并可高效扩展至极高分辨率,为高质量几何精修提供高效可拓展的解决方案。
原文摘要 · Abstract (English)
Joint estimation of surface normals and depth is essential for holistic 3D scene understanding, yet high-resolution prediction remains difficult due to the trade-off between preserving fine local detail and maintaining global consistency. To address this challenge, we propose the Ultra Resolution Geometry Transformer (URGT), which adapts the Visual Geometry Grounded Transformer (VGGT) into a unified multi-patch transformer for monocular high-resolution depth--normal estimation. A single high-resolution image is partitioned into patches that are augmented with coarse depth and normal priors from pre-trained models, and jointly processed in a single forward pass to predict refined geometric outputs. Global coherence is enforced through cross-patch attention, which enables long-range geometric reasoning and seamless propagation of information across patches within a shared backbone. To further enhance spatial robustness, we introduce a GridMix patch sampling strategy that probabilistically samples grid configurations during training, improving inter-patch consistency and generalization. Our method achieves state-of-the-art results on UnrealStereo4K, jointly improving depth and normal estimation, reducing AbsRel from 0.0582 to 0.0291, RMSE from 2.17 to 1.31, and lowering mean angular error from 23.36 degrees to 18.51 degrees, while producing sharper and more stable geometry. The proposed multi-patch framework also demonstrates strong zero-shot and cross-domain generalization and scales effectively to very high resolutions, offering an efficient and extensible solution for high-quality geometry refinement.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。