大模型与小模型协作,让机器人更懂指令、走得更稳。
CLASH: Collaborative Large-Small Hierarchical Framework for Continuous Vision-and-Language Navigation
- 大模型反思推理,小模型实时响应,双轨协同决策。
- 在未见过的测试场景中,成功率和路径效率均达当前最优。
- 适合需要真实部署的智能导航系统开发者参考。
视觉语言导航(VLN)要求机器人根据自然语言指令在无地图环境下导航。尽管大型视觉语言模型具备强大推理能力,但在特定任务上仍逊于全景小模型。为此,我们提出CLASH(协同大-小分层框架),整合反应式小模型规划器(RSMP)与反思式大模型推理器(RLMR)。RSMP采用基于因果学习的双分支架构以增强泛化能力,而RLMR利用全景视觉提示与思维链推理实现可解释的空间理解。我们进一步设计了不确定性感知协同机制(UCM),动态融合两模型决策。针对障碍物避让,在仿真中用可学习的点目标策略替代规则控制器;在真实部署中,设计基于激光雷达聚类的导航点生成模块,并搭配在线SLAM局部控制器。CLASH在VLN-CE排行榜上取得第一,显著提升测试未见场景下的成功率(SR)与路径相似率(SPL)。真实世界实验验证其强鲁棒性,证明该方法在仿真与实际部署中的有效性。
原文摘要 · Abstract (English)
Vision-and-Language Navigation (VLN) requires robots to follow natural language instructions and navigate complex environments without prior maps. While recent vision-language large models demonstrate strong reasoning abilities, they often underperform task-specific panoramic small models in VLN tasks. To address this, we propose CLASH (Collaborative Large-Small Hierarchy), a VLN-CE framework that integrates a reactive small-model planner (RSMP) with a reflective large-model reasoner (RLMR). RSMP adopts a causal-learning-based dual-branch architecture to enhance generalization, while RLMR leverages panoramic visual prompting with chain-of-thought reasoning to support interpretable spatial understanding and navigation. We further introduce an uncertainty-aware collaboration mechanism (UCM) that adaptively fuses decisions from both models. For obstacle avoidance, in simulation, we replace the rule-based controller with a fully learnable point-goal policy, and in real-world deployment, we design a LiDAR-based clustering module for generating navigable waypoints and pair it with an online SLAM-based local controller. CLASH achieves state-of-the-art (SoTA) results (ranking 1-st) on the VLN-CE leaderboard, significantly improving SR and SPL on the test-unseen set over the previous SoTA methods. Real-world experiments demonstrate CLASH's strong robustness, validating its effectiveness in both simulation and deployment scenarios.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。