基于Transformer的舱内相机位姿估计模型,实现单步精准定位。
InCaRPose: In-Cabin Relative Camera Pose Estimation Model and Dataset
- 用冻结的DINOv3特征与Transformer解码器捕捉视图间几何关系。
- 单次推理即得真实尺度平移,误差低于15厘米,旋转误差小于2度。
- 专为畸变严重的鱼眼镜头设计,适用于车载监控等安全场景。
相机外参标定是计算机视觉的基础任务。然而,在受限且高度畸变的舱内汽车监控(ICAM)环境中,精确的相对位姿估计仍具挑战性。我们提出InCaRPose,一种基于Transformer的架构,用于图像对之间的鲁棒相对位姿预测,可用于相机外参标定。通过利用DINOv3等冻结主干特征和Transformer解码器,模型有效捕捉参考视图与目标视图间的几何关系。与传统方法不同,本方法在单次推理中即可实现物理合理安装范围内的绝对度量尺度平移,这对需要真实世界距离的安全感知至关重要。我们针对汽车内饰中高度畸变的鱼眼相机问题,仅使用合成数据进行训练。模型无需依赖完全相同的相机内参即可泛化至真实舱内环境,并在公开的7-Scenes数据集上表现优异。尽管训练数据有限,InCaRPose在旋转和平移上均保持高精度,即使使用ViT-Small主干也能实现实时性能,适用于监督自动驾驶中的驾驶员监控等时序关键任务。我们发布了包含高度畸变车舱图像的真实世界In-Cabin-Pose测试数据集及代码,详见https://github.com/felixstillger/InCaRPose。
原文摘要 · Abstract (English)
Camera extrinsic calibration is a fundamental task in computer vision. However, precise relative pose estimation in constrained, highly distorted environments, such as in-cabin automotive monitoring (ICAM), remains challenging. We present InCaRPose, a Transformer-based architecture designed for robust relative pose prediction between image pairs, which can be used for camera extrinsic calibration. By leveraging frozen backbone features such as DINOv3 and a Transformer-based decoder, our model effectively captures the geometric relationship between a reference and a target view. Unlike traditional methods, our approach achieves absolute metric-scale translation within the physically plausible adjustment range of in-cabin camera mounts in a single inference step, which is critical for ICAM, where accurate real-world distances are required for safety-relevant perception. We specifically address the challenges of highly distorted fisheye cameras in automotive interiors by training exclusively on synthetic data. Our model is capable of generalization to real-world cabin environments without relying on the exact same camera intrinsics and additionally achieves competitive performance on the public 7-Scenes dataset. Despite having limited training data, InCaRPose maintains high precision in both rotation and translation, even with a ViT-Small backbone. This enables real-time performance for time-critical inference, such as driver monitoring in supervised autonomous driving. We release our real-world In-Cabin-Pose test dataset consisting of highly distorted vehicle-interior images and our code at https://github.com/felixstillger/InCaRPose.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。