面对多个有效答案时，投票会失效，该研究提出用因果逻辑验证选出最优解。
When Many Answers Are Valid, Voting Fails: Symbolic Verification for Best-of-K Causal Reasoning in LLMs
- 基于因果图准则设计符号验证器，不依赖训练和参考答案。
- 在多有效答案场景下准确率提升至42.1%，远超传统方法的30%左右。
- 适用于需构建因果图的文本推理，支持快速部署于CPU上。
自洽性假设采样推理路径中出现频率最高的答案最可靠，但在因果推理中常失效：样本可能重复相同混淆错误，且有效答案分散，导致无效答案因多数票胜出。本文提出CALVER（Causal Axiom-Level VERification），一种无需训练的符号验证器，通过检验佩尔因果标准——如d-分离、后门调整与干预——对结构化推理路径评分，并选择得分最高者，无需参考答案。在CLEAR数据集的find-one-valid查询任务中，当存在多个图合法答案时，CALVER达到42.1%准确率，而众数法、奖励模型、LLM判官及模型置信度均维持在约30%。即使将判官扩展至720亿参数，差距仍无法弥合。在经审计的纯净核心子集上，21个图合法选择中有11个不同于基准答案，但仍满足查询谓词。该优势随采样预算增加而扩大，且在十个公开贝叶斯网络、另一模型族以及需从文本构建图的设定中均复现。此外，CALVER在处理阈值平均治疗效应决策时优于精确真值，可泛化至真值表校验逻辑，单个候选评估仅需毫秒级时间，运行于CPU。CALVER只需提供或从文本构建因果结构，只要结构成立，即可通过因果有效性聚合选择。
原文摘要 · Abstract (English)
Self-consistency assumes the most frequent answer among sampled reasoning traces is the most reliable, but this can fail in causal reasoning: samples often repeat the same confounding error, and votes fragment across multiple valid answers, letting an invalid answer win despite a valid minority trace. We introduce CALVER (Causal Axiom-Level VERification), a training-free symbolic verifier that scores structured traces against Pearl's causal criteria, including -separation, backdoor adjustment, and intervention, and selects the highest-scoring candidate without consulting a reference answer. On CLEAR find-one-valid queries that admit multiple graph-valid answers, CALVER reaches 42.1% where plurality, a reward model, an LLM judge, and model confidence remain near 30% on identical frozen pools. Scaling the judge to 72B does not close the gap. In an audited clean-core subset, 11 of 21 graph-valid CALVER selections differ from the benchmark's listed answer while still satisfying the requested predicate. The advantage widens with the sampling budget and reproduces across ten published Bayesian networks, a second model family, and settings where the model must build the graph from text. CALVER also improves thresholded average-treatment-effect decisions against exact ground truth, generalizes to logic under a truth-table checker, and scores each candidate in milliseconds on CPU. CALVER needs only a causal structure, supplied outright or built from the text; wherever that holds, selection can aggregate via causal validity.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。