用视觉Transformer实现航天器实时位姿估计,无需复杂迭代算法。
FastPose-ViT: A Vision Transformer for Real-Time Spacecraft Pose Estimation
- 基于ViT直接回归6自由度位姿,避免传统PnP的计算开销。
- 在SPEED数据集上性能接近顶尖PnP方法,精度达1.2°/0.05m。
- 模型可量化部署于边缘设备,实测帧率高达33 FPS。
从单张图像中估算航天器的6自由度(6DoF)位姿对于在轨服务和空间碎片清除等自主操作至关重要。现有最先进方法通常依赖计算密集型的基于透视n点(PnP)的算法,难以在资源受限的边缘设备上实时运行。为此,我们提出FastPose-ViT,一种基于视觉变换器(ViT)的架构,可直接回归6DoF位姿。该方法处理目标边界框裁剪后的图像,并引入一种新颖的数学形式,将局部预测映射回全图尺度。该形式基于投影几何原理和“表观旋转”概念,模型先预测表观旋转矩阵,再经校正得到真实姿态。实验表明,该方法优于其他非PnP策略,在SPEED数据集上的表现与最先进的PnP方法相当。此外,我们通过量化并部署到功耗受限的边缘硬件验证了其实际适用性:在NVIDIA Jetson Orin Nano上,串行执行时每帧延迟约75毫秒,阶段并发调度时吞吐量可达33帧/秒。
原文摘要 · Abstract (English)
Estimating the 6-degrees-of-freedom (6DoF) pose of a spacecraft from a single image is critical for autonomous operations like in-orbit servicing and space debris removal. Existing state-of-the-art methods often rely on iterative Perspective-n-Point (PnP)-based algorithms, which are computationally intensive and ill-suited for real-time deployment on resource-constrained edge devices. To overcome these limitations, we propose FastPose-ViT, a Vision Transformer (ViT)-based architecture that directly regresses the 6DoF pose. Our approach processes cropped images from object bounding boxes and introduces a novel mathematical formalism to map these localized predictions back to the full-image scale. This formalism is derived from the principles of projective geometry and the concept of "apparent rotation", where the model predicts an apparent rotation matrix that is then corrected to find the true orientation. We demonstrate that our method outperforms other non-PnP strategies and achieves performance competitive with state-of-the-art PnP-based techniques on the SPEED dataset. Furthermore, we validate our model's suitability for real-world space missions by quantizing it and deploying it on power-constrained edge hardware. On the NVIDIA Jetson Orin Nano, our end-to-end pipeline achieves a latency of ~75 ms per frame under sequential execution, and a non-blocking throughput of up to 33 FPS when stages are scheduled concurrently.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。