arXiv:2603.28896cs.CV2026-03被引 3

让3D重建模型直接用鱼眼镜头图像,不降精度还无需鱼眼数据。

Fisheye3R: Adapting Unified 3D Feed-Forward Foundation Models to Fisheye Lenses

  • 用无标签透视图自监督+零鱼眼数据微调,适配鱼眼输入
  • 在3个主流模型上提升鱼眼图像的位姿、深度等估计精度
  • 适合做车载/机器人视觉的开发者,尤其缺鱼眼标注时

面向多视角三维重建的前馈基础模型通常在透视图像数据集上训练,当测试于广角图像(如鱼眼相机)时性能下降。这源于非线性投影导致像素空间分布变化。尽管训练时加入鱼眼图像可缓解问题,但真实鱼眼数据和标注远少于透视图像,限制泛化能力。为此,我们提出Fisheye3R,一种新适配框架,使多视角3D重建基础模型能原生支持鱼眼输入,且在透视图像上保持性能。针对鱼眼数据稀缺,引入灵活学习策略:仅用无标签透视图进行自监督适应,或完全不依赖鱼眼数据的有监督适应。在三个基础模型(VGGT、π³、MapAnything)上的实验证明,该方法显著提升鱼眼图像的相机位姿、深度图、点云和视场估计效果。代码已开源:https://github.com/android-xr/fisheye3r。

原文摘要 · Abstract (English)

Feed-forward foundation models for multi-view 3-dimensional (3D) reconstruction have been trained on large-scale datasets of perspective images; when tested on wide field-of-view images, e.g., from a fisheye camera, their performance degrades. This degradation arises from changes in spatial arrangements of pixels induced by the non-linear projection model that maps 3D points onto the 2D image plane. While one may surmise that training on fisheye images would resolve this problem, there are far fewer fisheye images with ground truth than perspective images, which limits generalization. To enable inference on imagery exhibiting high radial distortion, we propose \textit{Fisheye3R}, a novel adaptation framework that extends these multi-view 3D reconstruction foundation models to natively accommodate fisheye inputs without performance regression on perspective images. To address the scarcity of fisheye images and ground truth, we introduce flexible learning schemes that support self-supervised adaptation using only unlabeled perspective images and supervised adaptation without any fisheye training data. Extensive experiments across three foundation models, including VGGT, $π^3$, and MapAnything, demonstrate that our approach consistently improves camera pose, depth, point map, and field-of-view estimation on fisheye images. Code is available at https://github.com/android-xr/fisheye3r.

3D重建鱼眼图像自监督模型适配

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。