arXiv:2601.21157cs.AIcs.CL2026-01被引 1

解决大模型财务推理中的算术幻觉问题,提升金融计算可靠性。

Bridging the Arithmetic Gap: The Cognitive Complexity Benchmark and Financial-PoT for Robust Financial Reasoning

  • 构建三维度金融推理评估框架,精准定位复杂任务中的错误根源。
  • 提出双阶段神经符号方法,将语义理解与计算分离,准确率提升至67.3%。
  • 适合金融、审计等需高精度计算的领域应用,尤其适合对可靠性要求高的场景。

尽管大语言模型在语义任务上表现优异,但在财务量化推理方面存在严重瓶颈,常出现‘算术幻觉’和一种我们称为‘认知崩溃’的系统性失败模式。为严格量化该现象,我们提出了认知复杂度基准(CCB),基于95份真实中国A股年报构建数据集。不同于传统数据集,CCB将财务问题分为数据来源、映射难度和结果单位三个维度,可精准诊断高认知负荷下的推理退化。为此,我们提出迭代双阶段Financial-PoT框架,该神经符号架构通过严格解耦:先提取语义变量与逻辑结构,再将计算交由迭代自校正的Python沙箱执行,确保确定性。在CCB上的评估显示,标准链式思维在复杂任务中失效,而本方法显著提升鲁棒性,使Qwen3-235B模型平均准确率从59.7%提升至67.3%,在高复杂度任务中最高实现10倍性能增益。结果表明,架构解耦是提升金融推理可靠性的关键,为需语义与计算强对齐的高精度领域提供了可迁移的设计思路。

原文摘要 · Abstract (English)

While Large Language Models excel at semantic tasks, they face a critical bottleneck in financial quantitative reasoning, frequently suffering from "Arithmetic Hallucinations" and a systemic failure mode we term "Cognitive Collapse". To strictly quantify this phenomenon, we introduce the Cognitive Complexity Benchmark (CCB), a robust evaluation framework grounded in a dataset constructed from 95 real-world Chinese A-share annual reports. Unlike traditional datasets, the CCB stratifies financial queries into a three-dimensional taxonomy, Data Source, Mapping Difficulty, and Result Unit, enabling the precise diagnosis of reasoning degradation in high-cognitive-load scenarios. To address these failures, we propose the Iterative Dual-Phase Financial-PoT framework. This neuro-symbolic architecture enforces a strict architectural decoupling: it first isolates semantic variable extraction and logic formulation, then offloads computation to an iterative, self-correcting Python sandbox to ensure deterministic execution. Evaluation on the CCB demonstrates that while standard Chain-of-Thought falters on complex tasks, our approach offers superior robustness, elevating the Qwen3-235B model's average accuracy from 59.7\% to 67.3\% and achieving gains of up to 10-fold in high-complexity reasoning tasks. These findings suggest that architectural decoupling is a critical enabling factor for improving reliability in financial reasoning tasks, providing a transferable architectural insight for precision-critical domains that require tight alignment between semantic understanding and quantitative computation.

金融推理算术幻觉神经符号大模型评估

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。