通过优化提示设计,让小模型在导航中更懂社交规范。
Probing Prompt Design for Socially Compliant Robot Navigation with Vision Language Models
- 设计三类系统引导提示和三种激励框架,提升小模型决策能力。
- 对比实验发现,与人类竞争提示使性能最佳,直接微调对动作准确率帮助有限。
- 提示设计显著提升动作准确性,说明其核心作用是约束决策而非增强表征。
语言模型在社交机器人导航中的应用日益广泛,但现有基准普遍忽视有原则的提示设计。这一问题在实践中尤为突出,因为许多系统依赖小型视觉语言模型(VLMs)以保证效率。相比大型语言模型,小型VLM决策能力较弱,因此有效的提示设计至关重要。受人类学习与动机认知理论启发,本文从两个维度研究提示设计:系统引导(动作导向、推理导向、感知-推理提示)与动机框架(与人类、其他AI或自身过去版本竞争)。在两个社交合规导航数据集上的实验揭示三项关键发现:第一,对于未微调的GPT-4o,与人类竞争表现最佳,与其他AI竞争最差;对于微调模型,与自身过去版本竞争效果最强,其次为与人类竞争,且性能受提示设计、模型选择与数据集特征耦合效应影响。第二,不当的系统提示设计会显著降低性能,甚至低于直接微调。第三,直接微调虽显著提升感知、预测与推理等语义级指标,但对动作准确率提升有限;而本文提出的系统提示在动作准确率上带来更大比例提升,表明其主要作用是决策层约束而非表征增强。
原文摘要 · Abstract (English)
Language models are increasingly used for social robot navigation, yet existing benchmarks largely overlook principled prompt design for socially compliant behavior. This limitation is particularly relevant in practice, as many systems rely on small vision language models (VLMs) for efficiency. Compared to large language models, small VLMs exhibit weaker decision-making capabilities, making effective prompt design critical for accurate navigation. Inspired by cognitive theories of human learning and motivation, we study prompt design along two dimensions: system guidance (action-focused, reasoning-oriented, and perception-reasoning prompts) and motivational framing, where models compete against humans, other AI systems, or their past selves. Experiments on two socially compliant navigation datasets reveal three key findings. First, for non-finetuned GPT-4o, competition against humans achieves the best performance, while competition against other AI systems performs worst. For finetuned models, competition against the model's past self yields the strongest results, followed by competition against humans, with performance further influenced by coupling effects among prompt design, model choice, and dataset characteristics. Second, inappropriate system prompt design can significantly degrade performance, even compared to direct finetuning. Third, while direct finetuning substantially improves semantic-level metrics such as perception, prediction, and reasoning, it yields limited gains in action accuracy. In contrast, our system prompts produce a disproportionately larger improvement in action accuracy, indicating that the proposed prompt design primarily acts as a decision-level constraint rather than a representational enhancement.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。