让大模型的空间推理更可信，通过符号化分析提升结论可靠性判断。
SymboUQ: Symbolic Uncertainty Quantification for Spatial Reasoning in LLMs
- 用符号化方法区分命题可表达性与语义确定性，评估推理可信度。
- 在五个基准上比最强基线提升8%的AUROC，降低7%的Brier损失。
- 适合关注大模型推理可靠性、需增强可信决策的应用场景。
尽管大语言模型（LLMs）能生成流畅的空间推理过程，但其中间关系可能无法支持最终结论，导致基于标记的置信度不足以衡量最终答案的可靠性。现有形式化验证器虽提供更强的语义证据，但适用性有限：一个解析后的命题未必能得出明确的语义判断。为此，我们提出SymboUQ，一种符号化不确定性量化框架，通过区分命题的可符号化性（能否在验证器的形式语言中表示）与语义确定性（执行是否产生蕴含或矛盾结果而非未知或不可评估），从推理轨迹估计最终答案的可靠性。SymboUQ包含：(i) 布局审计器，执行有序空间命题并提取可行性、冲突与修复证据；(ii) 无标签确定性轮廓，刻画有效可执行覆盖范围；(iii) 确定性感知可靠性组合器，根据验证器适用性整合基于约束、表示和解码的得分。在四个冻结的LLM主干与五个空间推理基准上的实验表明，SymboUQ相较最强基线实现约8%的相对AUROC提升和7%的类别平衡Brier损失降低。
原文摘要 · Abstract (English)
Although large language models (LLMs) can produce fluent spatial reasoning traces, their intermediate relations may fail to support the final conclusion, making token-level confidence insufficient for final-answer reliability estimation. Existing formal verifiers provide stronger semantic evidence, but their applicability is partial: a parsed claim need not yield a definite semantic verdict. To address this issue, we introduce SymboUQ, a symbolic uncertainty quantification framework that estimates final-answer reliability from reasoning traces by distinguishing symbolizability, whether a claim can be represented in the verifier's formal language, from semantic determinacy, whether its execution yields an entailed or contradicted verdict rather than an unknown or not-evaluable outcome. SymboUQ comprises (i) a Layout Auditor that executes ordered spatial claims and extracts feasibility, conflict, and repair evidence; (ii) a label-free Determinacy Profile that characterizes effective executable coverage; and (iii) a Determinacy-Aware Reliability Composer that integrates constraint-based, representation-based, and decoding-based scores according to verifier applicability. Extensive experiments on five spatial reasoning benchmarks with four frozen LLM backbones show that SymboUQ achieves approximately an 8% relative improvement in AUROC and a 7% relative reduction in class-balanced Brier loss over the strongest baseline.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。