仅用预训练视觉语言嵌入实现机器人导航，验证其可行性与局限性
Can Pretrained Vision-Language Embeddings Alone Guide Robot Navigation?
- 直接在冻结的视觉语言嵌入上训练行为克隆策略
- 导航成功率74%，需步数为专家的3.2倍
- 适合评估基础模型在具身任务中的实用边界
基础模型通过无需特定任务训练即可提供丰富的语义表征，革新了机器人技术。尽管许多方法将预训练视觉语言模型（VLMs）与专用导航架构结合，但核心问题仍未解决：仅靠这些预训练嵌入能否在不进行微调或添加专用模块的情况下成功引导导航？我们提出一种极简框架，通过在特权专家收集的演示数据上直接对冻结的视觉语言嵌入训练行为克隆策略，解耦该问题。该方法在语言指定目标导航中达到74%的成功率，相比状态感知专家的100%有所差距，但平均所需步数多出3.2倍。这一性能差距表明，预训练嵌入虽能有效支持基本的语言定位，但在长时程规划和空间推理方面存在不足。本研究提供了关键的实证基线，揭示了将基础模型作为即插即用表征应用于具身任务的能力与局限，为资源受限场景下系统复杂性与性能之间的权衡决策提供了重要参考。代码已开源：https://github.com/oadamharoon/text2nav
原文摘要 · Abstract (English)
Foundation models have revolutionized robotics by providing rich semantic representations without task-specific training. While many approaches integrate pretrained vision-language models (VLMs) with specialized navigation architectures, the fundamental question remains: can these pretrained embeddings alone successfully guide navigation without additional fine-tuning or specialized modules? We present a minimalist framework that decouples this question by training a behavior cloning policy directly on frozen vision-language embeddings from demonstrations collected by a privileged expert. Our approach achieves a 74% success rate in navigation to language-specified targets, compared to 100% for the state-aware expert, though requiring 3.2 times more steps on average. This performance gap reveals that pretrained embeddings effectively support basic language grounding but struggle with long-horizon planning and spatial reasoning. By providing this empirical baseline, we highlight both the capabilities and limitations of using foundation models as drop-in representations for embodied tasks, offering critical insights for robotics researchers facing practical design tradeoffs between system complexity and performance in resource-constrained scenarios. Our code is available at https://github.com/oadamharoon/text2nav
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。