用大模型让无人机在复杂环境自动精准降落,避免碰撞。
LLM-Land: Large Language Models for Context-Aware Drone Landing
- 结合视觉语言模型与模型预测控制,实现动态环境下的语义感知
- 对行人、车辆分别设定3米、5米的安全缓冲区,显著减少近撞事件
- 适合应急救援、灾后搜救等复杂场景的无人机自主任务
自主降落对执行紧急配送、灾后响应等大规模任务的无人机至关重要。通过实现充电平台上的自动对接,可保障持续运行并大幅提升任务续航。然而,传统方法在动态非结构化环境中常因语义感知能力有限且依赖固定安全距离而表现不佳。为此,我们提出一种融合大语言模型(LLM)与模型预测控制(MPC)的混合框架。首先利用视觉-语言编码器(如BLIP)将实时图像转化为简明文本描述;再由轻量级LLM(如Qwen 2.5 1.5B或LLaMA 3.2 1B)结合检索增强生成(RAG),识别场景元素并推断上下文感知的安全缓冲区(行人3米,车辆5米)。这些语义标志与危险区域信息输入MPC模块,实现实时轨迹重规划,在保持高着陆精度的同时规避碰撞。我们在ROS-Gazebo仿真环境中验证了该框架,结果表明其显著降低了与动态障碍物的近撞事件,同时在杂乱环境中仍能实现精准着陆。
原文摘要 · Abstract (English)
Autonomous landing is essential for drones deployed in emergency deliveries, post-disaster response, and other large-scale missions. By enabling self-docking on charging platforms, it facilitates continuous operation and significantly extends mission endurance. However, traditional approaches often fall short in dynamic, unstructured environments due to limited semantic awareness and reliance on fixed, context-insensitive safety margins. To address these limitations, we propose a hybrid framework that integrates large language model (LLMs) with model predictive control (MPC). Our approach begins with a vision-language encoder (VLE) (e.g., BLIP), which transforms real-time images into concise textual scene descriptions. These descriptions are processed by a lightweight LLM (e.g., Qwen 2.5 1.5B or LLaMA 3.2 1B) equipped with retrieval-augmented generation (RAG) to classify scene elements and infer context-aware safety buffers, such as 3 meters for pedestrians and 5 meters for vehicles. The resulting semantic flags and unsafe regions are then fed into an MPC module, enabling real-time trajectory replanning that avoids collisions while maintaining high landing precision. We validate our framework in the ROS-Gazebo simulator, where it consistently outperforms conventional vision-based MPC baselines. Our results show a significant reduction in near-miss incidents with dynamic obstacles, while preserving accurate landings in cluttered environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。