无需真实标签，通过2D对应点实现3D视觉模型测试时几何一致性自适应。
Self-Geometry: GT-Free and Plug-and-Play Test-Time Adaptation for Geometrically Consistent 3D Vision Foundation Models
- 利用2D像素对应点作为伪标签，显式施加多视图几何约束。
- 在6个模型、4个数据集上均提升姿态与几何估计精度。
- 插件式设计，兼容多种模型，适合部署场景下的实时优化。
近期视觉基础模型（VFMs）可在单次前向传播中预测深度、相机位姿和点云，无需逐场景优化，具备强大泛化能力。然而，由于预训练阶段未强制多视图几何一致性（如通过束调整），可能导致输出不一致。现有方法依赖模型输出的隐式自一致性信号，在预训练模型表现较差的场景中提升有限。为此，本文提出Self-Geometry，一种插件式测试时自适应框架，直接利用2D像素对应点作为伪真值，显式施加多视图几何约束。该方法包含几何解耦优化（结合多视图一致性与极线一致性损失及梯度解耦以避免梯度冲突）、基于SO(3)测地距离的帧角邻域采样器（轻量施加约束），以及基于LoRA的轻量级TTA适配。实验在6个模型（VGGT, $π^3$, DA3-Giant/Large/Base/Small）和4个基准（7Scenes, ETH3D, ScanNet++, HiRoom）上验证，显著提升姿态与几何估计性能。
原文摘要 · Abstract (English)
Recent Vision Foundation Models (VFMs) predict depth, camera pose, and pointmap in a single forward pass without per-scene optimization, achieving strong generalization. However, enforcing explicit multi-view geometric consistency, e.g., through bundle adjustment, is computationally costly and is thus not imposed during VFM pretraining, so such inconsistency can arise. To address this, implicit self-consistency derived from model outputs (e.g., pointmaps, features), though enforced at test-time in prior work, delivers inherently limited performance gain, especially on scenes where the pretrained VFM is highly inaccurate. In contrast to this implicit signal, we propose Self-Geometry, a plug-and-play test-time adaptation pipeline that directly imposes explicit multi-view geometric constraints using 2D pixel correspondences as pseudo ground-truth. Our proposed Self-Geometry consists of Geometric Disentanglement Optimization, which combines Multi-View Consistency and Epipolar Consistency losses with Gradient Disentanglement to prevent gradient conflict; Frame Angular-Neighbor, a view sampler based on SO(3) geodesic distances for lightly imposing these constraints; and Lightweight TTA, which adapts VFMs via LoRA. Our method achieves consistent improvements in both pose and geometry estimation across six VFMs (VGGT, $π^3$, DA3-Giant/Large/Base/Small) and four benchmarks (7Scenes, ETH3D, ScanNet++, HiRoom).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。