arXiv:2603.23867cs.LGcs.AI2026-03被引 1

提出新方法VLC,让视觉语言模型在分布变化下仍能稳定推理。

Can VLMs Reason Robustly? A Neuro-Symbolic Investigation

  • 用符号电路解耦感知与推理,精准执行逻辑规则
  • 在三种任务上,外分布数据准确率显著高于现有方法
  • 适合需要可靠逻辑推理的场景,如医疗诊断、自动驾驶

视觉语言模型(VLMs)被广泛应用于各类推理任务,但其在分布偏移下的鲁棒性尚不明确。本文研究了协变量偏移:感知输入分布变化,但底层预测规则不变。通过视觉演绎推理任务,发现基于梯度的端到端微调虽在分布内表现优异,却无法泛化至分布外,表明微调未必能获得稳定的推理功能。为此,提出神经符号方法VLC,将VLM的概念识别与电路式符号推理结合。具体而言,任务规则被编译为符号程序(电路),在VLM识别出的对象概念上精确执行。在三个具有不同规则集的简单视觉演绎推理任务上,VLC在分布外数据上始终优于其他推理范式。代码已开源。

原文摘要 · Abstract (English)

Vision-Language Models (VLMs) have been applied to a wide range of reasoning tasks, yet it remains unclear whether they can reason robustly under distribution shifts. In this paper, we study covariate shifts in which the perceptual input distribution changes while the underlying prediction rules do not. To investigate this question, we consider visual deductive reasoning tasks, where a model is required to answer a query given an image and logical rules defined over the object concepts in the image. Empirically, we find that VLMs fine-tuned through gradient-based end-to-end training can achieve high in-distribution accuracy but fail to generalize under such shifts, suggesting that fine-tuning does not reliably induce the underlying reasoning function. This motivates a neuro-symbolic perspective that decouples perception from reasoning. However, we further observe that recent neuro-symbolic approaches that rely on black-box components for reasoning can still exhibit inconsistent robustness across tasks. To address this issue, we propose VLC, a neuro-symbolic method that combines VLM-based concept recognition with circuit-based symbolic reasoning. In particular, task rules are compiled into a symbolic program, specifically a circuit, which executes the rules exactly over the object concepts recognized by the VLM. Experiments on three simple visual deductive reasoning tasks with distinct rule sets show that VLC consistently achieves higher task accuracy on out-of-distribution data than other reasoning paradigms. Code is available at https://github.com/uiuctml/VLC.

神经符号视觉推理鲁棒性符号计算

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。