arXiv:2607.11414cs.CL2026-07

通过分析模型内部状态,可有效识别金融问答中自信的错误答案。

Confidently Wrong: Detecting Hallucinations in Financial Question Answering from LLM Internal States

  • 利用残差流中的激活值训练线性探测器,捕捉模型内在信心
  • 在FinQA上,15%-23%的自信回答实际错误,探测器准确率超基线
  • 适合金融领域高风险场景下的模型输出审核与人工复核

大型语言模型在金融应用中最为危险的情况是自信地给出错误答案。含糊或不确定的回答容易被察觉,而自信的错误却无声无息地影响下游决策。本文研究能否从模型内部激活值中可靠检测这类‘自信幻觉’,以及内部状态是否蕴含超出输出的信息。我们在真实财报数据构建的两个问答基准(FinQA 和 TAT-QA)上,对残差流进行线性探测。行为置信度以同一问题8次重采样结果的一致性衡量;探测效果与词元概率、模型自评真/假等基线方法对比。结果显示,在自信回答中,15%-23%在FinQA上为错误;探测器在Qwen3-8B、Llama-3.1-8B、Gemma-2-9B上均表现更优,AUROC达0.68-0.77,优于基线的0.55-0.63。结果表明,探测可作为金融高风险场景下低成本的优先筛查机制,用于引导模型输出进入人工审查流程。

原文摘要 · Abstract (English)

Large language models (LLMs) in financial applications fail most consequentially when they are confidently wrong. Hedged, uncertain answers invite scrutiny, whereas confident errors silently degrade downstream decisions without warning. We ask how reliably such confidently wrong answers, or confident hallucinations, can be detected from a model's internal activations, and whether those activations carry information beyond its observable outputs. We train linear probes on the residual stream and evaluate them on two established question-answering (QA) benchmarks built from real filings, FinQA and TAT-QA. Behavioral confidence is measured as the agreement among eight resampled answers to the same question, and probe effectiveness is compared against baselines, such as token log-probabilities and the model's own True/False self-assessment of its answer. Our findings show that among confident answers, those for which all eight resamples agree, 15-23% are wrong on FinQA. There the probes have a significant advantage over baseline methods in detecting hallucinations, holding 0.68-0.77 AUROC while the best baselines fall to 0.55-0.63, across Qwen3-8B, Llama-3.1-8B, and Gemma-2-9B. Our results suggest that probing can be a cost-effective triage mechanism for routing LLM answers to human review and quality control procedures in high-stakes financial applications.

金融AI幻觉检测模型可信度内部状态

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。