用虚拟海洋环境测试无人机视觉定位,降低成本。
Vision-in-the-loop Simulation for Deep Monocular Pose Estimation of UAV in Ocean Environment
- 基于高斯点云构建逼真虚拟海洋场景,支持多视角图像融合。
- 可在室内验证无人机在真实船甲板上的自主飞行与定位性能。
- 适合研究无人机视觉导航与自主控制的团队使用。
本文提出一种面向海上环境下无人机单目位姿估计的视觉闭环仿真系统。近期,基于Transformer架构的深度神经网络已成功实现无人机相对于科研船甲板的位姿估计,克服了基于GPS方法的诸多局限。然而,在真实海洋环境中验证该方案面临研究船舶资源稀缺和高昂操作成本的挑战。为此,我们利用最新发展的高斯点云技术(Gaussian splatting),通过将图像像素建模为三维空间中的高斯分布,从多个视角生成轻量级且高质量的三维视觉模型,构建了一个包含实地采集图像的逼真虚拟环境。该仿真系统支持在室内测试飞行动作,同时验证飞行软件、硬件及深度单目位姿估计方案的完整性。该方法为舰载无人机自主飞行的测试与验证提供了低成本解决方案,尤其适用于视觉感知与控制算法的研究。
原文摘要 · Abstract (English)
This paper proposes a vision-in-the-loop simulation environment for deep monocular pose estimation of a UAV operating in an ocean environment. Recently, a deep neural network with a transformer architecture has been successfully trained to estimate the pose of a UAV relative to the flight deck of a research vessel, overcoming several limitations of GPS-based approaches. However, validating the deep pose estimation scheme in an actual ocean environment poses significant challenges due to the limited availability of research vessels and the associated operational costs. To address these issues, we present a photo-realistic 3D virtual environment leveraging recent advancements in Gaussian splatting, a novel technique that represents 3D scenes by modeling image pixels as Gaussian distributions in 3D space, creating a lightweight and high-quality visual model from multiple viewpoints. This approach enables the creation of a virtual environment integrating multiple real-world images collected in situ. The resulting simulation enables the indoor testing of flight maneuvers while verifying all aspects of flight software, hardware, and the deep monocular pose estimation scheme. This approach provides a cost-effective solution for testing and validating the autonomous flight of shipboard UAVs, specifically focusing on vision-based control and estimation algorithms.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。