arXiv:2607.16727cs.AIcs.CV2026-07

让大模型推理时插入可验证的约束条件,防止错误蔓延。

Constraint-Anchored Reasoning Traces

论文配图:Constraint-Anchored Reasoning Traces
图 1 · 摘自论文原文
  • 在自然语言推理中穿插可机器验证的视觉约束,如物体数量
  • 错误检测后自动回溯,使错误雪崩率从65%降至14%
  • 适合需要高可靠性推理的场景,如医疗或自动驾驶

自回归多模态大模型存在错误雪崩问题:链式思维中早期一个错误会污染后续所有步骤。我们发现,在主流开源多模态大模型中,一旦首次出错,65%的情况会导致后续全部失败(称为雪崩率)。现有缓解方法或缺乏符号基础,或纠错过晚,或牺牲自然语言灵活性。本文提出约束锚定推理轨迹(CART),一种神经符号框架,训练模型在自然语言推理步骤间插入轻量级、可机器验证的符号约束(如count(red_objects) = 3)。通过融合学习的神经对齐头与布尔约束传播的双重机制,持续验证约束与视觉特征的一致性及逻辑自洽性。检测到矛盾时,回溯控制器终止生成并回退至上一一致状态,阻止错误传播。可变频率发射机制动态控制约束密度,避免轨迹膨胀。我们基于场景图生成真实约束,扩充GQA、CLEVR-CoGenT和VCR,构建21.8万条训练数据,使用LoRA微调LLaVA-NeXT和Qwen2-VL。在五个基准测试中,CART将雪崩率从0.65降至0.14,使GQA准确率比仅训练基线提升+4.6个百分点,并在POPE上达到89.1的F1值,推理开销不超过18%。

原文摘要 · Abstract (English)

Autoregressive multimodal large language models (MLLMs) suffer from error snowballing: a single incorrect inference early in a chainof-thought (CoT) trace corrupts all downstream reasoning. We find that in state-of-the-art open-source MLLMs, once the first error occurs, the reasoning cascades into failure across all remaining steps in 65% of such cases (a metric we term the snowball rate). Existing mitigations-sampling multiple chains, post-hoc self-verification, or full program synthesis-either lack symbolic grounding, catch errors too late, or sacrifice the flexibility of natural language reasoning. We propose Constraint-Anchored Reasoning Traces (CART), a neuro-symbolic framework that trains MLLMs to interleave natural language reasoning steps with symbolic constraint assertions: lightweight, machine-checkable statements about visual content (e.g., count(red_objects) = 3). A dual-pronged Constraint Propagation Module-combining a learned neural grounding head with Boolean Constraint Propagation-continuously verifies these anchors against extracted visual features and checks their mutual logical consistency. When a contradiction is detected, a backtrack controller halts generation and reverts to the last consistent checkpoint, preventing error propagation. A variable-frequency emission mechanism allows the model to adaptively control anchor density, avoiding trace bloat. We construct 218K training instances by augmenting GQA, CLEVR-CoGenT, and VCR with ground-truth constraint annotations derived from scene graphs, and fine-tune open-source MLLMs (LLaVA-NeXT, Qwen2-VL) via LoRA. On five benchmarks, CART reduces the snowball rate from 0.65 to 0.14, improves GQA accuracy by +4.6 percentage points over trainingonly baselines, and achieves 89.1 F1 on POPE-all with at most 18% inference overhead.

多模态推理错误控制神经符号约束传播

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。