arXiv:2609.05928cs.CLcs.AI2026-09

让大模型先验证再求解,能有效发现税务输入中的矛盾。

Solving versus Verifying: Catching Contradictions in Tax Reasoning Systems

论文配图:Solving versus Verifying: Catching Contradictions in Tax Reasoning Systems
图 1 · 摘自论文原文
  • 用验证任务检测输入矛盾,再通过自检门控控制输出
  • 模型在有矛盾时仍返回正确答案的比例为63%-76%
  • 只需一次额外验证调用,就能大幅提高容错能力

大型语言模型在结构良好的税务基准测试中正确计算税额的比率超过90%,可直接用于税务咨询与合规系统。但真实法律输入常存在缺失事实或事实矛盾。基准测试的高准确率无法反映模型在缺陷输入下的表现;若模型直接计算有缺陷输入,会自信地输出结果而无任何错误提示。我们研究了六种近期模型在SARA衍生的税务案例上,面对缺失事实和矛盾事实扰动的表现。最强模型在缺失事实时会放弃回答,但在注入矛盾时仍会输出原答案,正确率高达63%-76%且无冲突信号;而当被要求验证同一输入时,它们能识别出大多数矛盾。我们将此验证机制嵌入一个简单的矛盾门控:一次额外验证调用,一旦发现矛盾即拒绝输出。该方法在所有六种模型中均恢复了大部分因矛盾未中止的情况,干净输入准确率损失不超过约5个百分点,无需训练或外部工具。因此,高准确率并不能完全衡量可靠性,而求解器忽略的矛盾可通过一次自检低成本恢复。

原文摘要 · Abstract (English)

Large language models now compute correct tax liabilities on over 90% of well-formed cases in statutory benchmarks, which makes them candidates for the tax-advisory and compliance systems that consume such an answer directly. Real legal inputs, however, are frequently defective: required facts are missing, or stated facts contradict one another. Accuracy on clean benchmarks says nothing about how a model behaves then, and a system that computes straight through a defective input returns a confident number with no sign that anything is wrong. This raises two questions: does a model asked to solve a case abstain when the input is defective, and when it does not, can the same model catch the defect when asked instead to verify the input? We study six recent models on SARA-derived tax cases under missing-fact and contradictory-fact perturbations. The strongest models abstain when a fact is missing but compute through injected contradictions, returning the clean-input answer 63-76% of the time with no signal of the conflict; asked instead to verify the same input, they flag most of those contradictions. We wire that verification call into a simple contradiction gate: one extra call that abstains when the model reports a conflict. Across all six models it recovers most of the missed contradiction abstention at a clean-accuracy cost of at most about 5 percentage points, with no training and no external tooling. High accuracy on well-formed inputs is therefore an incomplete measure of reliability, and the detection the solver misses is cheaply recoverable with a single self-check.

税务AI模型验证自检机制纠错

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。