用伪代码引导推理，让视觉语言模型更可靠不乱编。
Pseudocode-Guided Structured Reasoning for Automating Reliable Inference in Vision-Language Models
- 通过结构化伪代码库和难度评估，动态选择推理路径。
- 在POPE和MMStar数据集上分别达到87.1%和68.0%准确率。
- 适合需要高可靠性推理的机器人自动化系统。
视觉语言模型（VLMs）正成为机器人自动化中高级推理的核心，使机器人能解析自然语言指令并感知环境。然而，其易产生幻觉的问题在决策中引入关键性失败，对物理部署的安全性和可靠性构成重大风险。这一挑战因现实任务的开放性而加剧——问题在难度和模态上差异巨大，需具备鲁棒且可适应的推理策略。为此，我们提出伪代码引导的结构化推理框架（PStar），通过自适应选择结构化伪代码推理路径，实现灵活、分步推理。我们设计了一组抽象推理函数，并构建了结构化伪代码库以表示模块化推理策略。关键在于，我们引入难度特征向量（DFV），使模型能评估问题复杂度并自适应选择合适策略，从而提升鲁棒性与可解释性。大量实验表明，PStar显著降低幻觉率，在POPE上达到87.1%、在MMStar上达到68.0%，超越GPT-4V表现。该方法为减少视觉语言错误提供了可验证机制，是推动更可信、确定性VLM在真实自动化系统中部署的关键一步，因为此类错误可能引发灾难性后果。
原文摘要 · Abstract (English)
Vision-Language Models (VLMs) are becoming the cornerstone of high-level reasoning for robotic automation, enabling robots to parse natural language commands and perceive their environments. However, their susceptibility to hallucinations introduces critical failures in decision-making, posing significant safety and reliability risks in physical deployments. This challenge is exacerbated by the open-ended nature of real-world tasks, where questions vary vastly in difficulty and modality, demanding robust and adaptable reasoning strategies. To tackle this, we propose the Pseudocode-guided Structured Reasoning framework (PStar), which adaptively selects structured pseudocode reasoning paths to help VLMs perform flexible and step-by-step reasoning. We first design a set of abstract reasoning functions and formulate a structured pseudocode library to represent modular reasoning strategies. Crucially, we design a Difficulty Feature Vector (DFV) that allows the model to assess question complexity and adaptively choose appropriate reasoning strategies-enhancing robustness and interpretability. Extensive experiments demonstrate that PStar significantly reduces hallucination rates, achieving state-of-the-art scores of 87.1% on POPE and 68.0% on MMStar, outperforming even GPT-4V. By providing a validated mechanism to reduce visual-language errors, PStar offers a critical step toward deploying more trustworthy and deterministic VLMs for real-world automated systems, where such errors can lead to catastrophic outcomes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。