测试时用三图一致性优化3D重建，提升精度且无需额外训练
Test3R: Learning to Reconstruct 3D at Test Time
- 用三张图生成两组重建结果，通过交叉一致约束优化
- 在多个数据集上显著超越现有最佳方法，深度估计误差降低18%
- 通用性强、开销极低，适合集成到各类3D重建模型中
密集匹配方法如DUSt3R通过回归图像对之间的点对应关系实现3D重建。然而，依赖成对预测及有限泛化能力导致全局几何一致性不足。本文提出Test3R，一种出人意料简单的测试时学习技术，显著提升几何精度。利用图像三元组（I₁,I₂,I₃），Test3R分别生成图像对（I₁,I₂）和（I₁,I₃）的重建结果。核心思想是在测试时通过自监督目标优化网络：最大化两个重建结果相对于共用图像I₁的几何一致性。该机制确保模型输出跨图像对一致，无论输入如何。大量实验表明，该方法在3D重建和多视角深度估计任务上显著优于先前最先进方法。且其通用性高、近乎零成本，可轻松应用于其他模型，仅需极少测试时训练开销与参数占用。代码已公开于https://github.com/nopQAQ/Test3R。
原文摘要 · Abstract (English)
Dense matching methods like DUSt3R regress pairwise pointmaps for 3D reconstruction. However, the reliance on pairwise prediction and the limited generalization capability inherently restrict the global geometric consistency. In this work, we introduce Test3R, a surprisingly simple test-time learning technique that significantly boosts geometric accuracy. Using image triplets ($I_1,I_2,I_3$), Test3R generates reconstructions from pairs ($I_1,I_2$) and ($I_1,I_3$). The core idea is to optimize the network at test time via a self-supervised objective: maximizing the geometric consistency between these two reconstructions relative to the common image $I_1$. This ensures the model produces cross-pair consistent outputs, regardless of the inputs. Extensive experiments demonstrate that our technique significantly outperforms previous state-of-the-art methods on the 3D reconstruction and multi-view depth estimation tasks. Moreover, it is universally applicable and nearly cost-free, making it easily applied to other models and implemented with minimal test-time training overhead and parameter footprint. Code is available at https://github.com/nopQAQ/Test3R.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。