用单目相机实现无人机低空环境的快速语义分割与深度估计
Co-SemDepth: Fast Joint Semantic Segmentation and Depth Estimation on Aerial Images
- 设计联合深度学习架构，同步预测语义分割和深度图
- 在NVIDIA Quadro P5000上达到20.2帧/秒，内存占用低
- 适用于需要实时感知的无人机自主导航场景
理解场景的几何与语义属性对自主导航至关重要，尤其在无人机（UAV）导航中更具挑战性。通过估计周围环境的深度和语义分割图可实现该目标，而为支持实际应用，该过程需尽可能接近实时。本文利用安装在空中机器人上的单目相机，在低空非结构化环境中同时预测深度图与语义分割图。提出一种联合深度学习架构，能够高精度且快速完成两项任务，并在MidAir和Aeroscapes基准数据集上验证有效性。该联合架构在性能上优于或媲美其他单任务及联合方法，可在单块NVIDIA Quadro P5000 GPU上实现20.2 FPS的推理速度，且内存开销小。训练与推理代码已开源：https://github.com/Malga-Vision/Co-SemDepth。
原文摘要 · Abstract (English)
Understanding the geometric and semantic properties of the scene is crucial in autonomous navigation and particularly challenging in the case of Unmanned Aerial Vehicle (UAV) navigation. Such information may be by obtained by estimating depth and semantic segmentation maps of the surrounding environment and for their practical use in autonomous navigation, the procedure must be performed as close to real-time as possible. In this paper, we leverage monocular cameras on aerial robots to predict depth and semantic maps in low-altitude unstructured environments. We propose a joint deep-learning architecture that can perform the two tasks accurately and rapidly, and validate its effectiveness on MidAir and Aeroscapes benchmark datasets. Our joint-architecture proves to be competitive or superior to the other single and joint architecture methods while performing its task fast predicting 20.2 FPS on a single NVIDIA quadro p5000 GPU and it has a low memory footprint. All codes for training and prediction can be found on this link: https://github.com/Malga-Vision/Co-SemDepth
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。