让无人机实时听懂指令安全导航，突破零样本飞行瓶颈
OnFly: Onboard Zero-Shot Aerial Vision-Language Navigation toward Safety and Efficiency
- 双代理架构分离目标生成与进度监控，稳定决策过程
- 任务成功率从26.4%提升至67.8%，支持长程可靠追踪
- 适合追求实时性与安全性的无人机自主导航研究者
航拍视觉语言导航（AVLN）使无人机能在复杂三维环境中遵循自然语言指令。然而现有零样本方法常面临单流视觉语言模型决策不稳定、长程进展监测不可靠，以及安全与效率的权衡问题。本文提出OnFly，一种完全在机端运行的实时零样本AVLN框架。OnFly采用共享感知双代理架构，将高频目标生成与低频进度监控解耦，从而稳定决策。通过混合关键帧-近期帧记忆机制，在保持KV缓存前缀稳定的同时保存全局轨迹上下文，实现可靠的长程监控并支持终止与恢复信号。此外，语义-几何验证器结合视觉语言模型特征与深度线索，修正目标以保证指令一致性和几何安全；回溯视野规划器在几何安全约束下生成无碰撞优化轨迹，兼顾安全与效率。仿真测试中，OnFly将任务成功率从26.4%提升至67.8%，超越最强基线；真实飞行实验验证了其在机端实时部署的可行性。代码将开源于https://github.com/Robotics-STAR-Lab/OnFly。
原文摘要 · Abstract (English)
Aerial vision-language navigation (AVLN) enables UAVs to follow natural-language instructions in complex 3D environments. However, existing zero-shot AVLN methods often suffer from unstable single-stream Vision-Language Model decision-making, unreliable long-horizon progress monitoring, and a trade-off between safety and efficiency. We propose OnFly, a fully onboard, real-time framework for zero-shot AVLN. OnFly adopts a shared-perception dual-agent architecture that decouples high-frequency target generation from low-frequency progress monitoring, thereby stabilizing decision-making. It further employs a hybrid keyframe-recent-frame memory to preserve global trajectory context while maintaining KV-cache prefix stability, enabling reliable long-horizon monitoring with termination and recovery signals. In addition, a semantic-geometric verifier refines VLM-predicted targets for instruction consistency and geometric safety using VLM features and depth cues, while a receding-horizon planner generates optimized collision-free trajectories under geometric safety constraints, improving both safety and efficiency. In simulation, OnFly improves task success from 26.4% to 67.8%, compared with the strongest state-of-the-art baseline, while fully onboard real-world flights validate its feasibility for real-time deployment. The code will be released at https://github.com/Robotics-STAR-Lab/OnFly
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。