AI在金融监管企业落地难，关键在于人工审查负担过重。
The Checking Problem: What must be true before AI ships in a regulated firm
- 通过72种配置测试，发现仅32种满足持续准确与可复现的生产标准。
- 要求AI提供来源和置信度，可将人工审查量从100%降至49%。
- 自检机制虽提升可靠性，但延迟增加2.3倍，且无法保证误差容忍度。
企业级AI项目停滞率高，却缺乏清晰解释。本文测量了这一现象背后的机制：在四类模型、三种工具配置下，对六种文书密集型工作流程进行三轮测试，共生成5,093个评分输出，覆盖72种配置。每种配置评估两次：一次为演示基准（单次正确），一次为生产基准（需持续准确、可复现、可追溯、带有效置信信号）。72种配置中57种通过演示基准，32种通过生产基准，生存率为56.1%。进一步计算各配置带来的审查负担：不提供置信度的工具需100%人工审查；若要求引用来源并标注置信度，审查量降至49%，且在20种配置中保持误差容忍度；增加自验证环节会使延迟增加2.3倍，审查量降至44%，但唯一未能维持误差容忍度。核心结论是，AI工作流价值不仅取决于准确率，更取决于人类需检查的比例，而后者可量化却极少被测量。
原文摘要 · Abstract (English)
Enterprise AI programmes stall at a rate that is widely quoted and poorly explained. This paper measures the mechanism. Six document-heavy workflows of the kind performed daily in regulated financial services were run across four model families and three tool configurations, three times each, producing 5,093 scored output elements across 72 configurations. Each configuration was assessed twice: against a demonstration bar, being a single correct run on a single case, and against a production bar requiring sustained accuracy, reproducibility across repeats, verifiable attribution, and a confidence signal that carries information. 57 of 72 configurations cleared the demonstration bar and 32 cleared the production bar, a survival rate of 56.1%. The paper then computes the review burden each configuration imposes, estimated out of sample rather than with hindsight. A tool that states no confidence requires review of 100% of its output, because it offers a reviewer no basis for triage. Requiring the tool to cite its sources and state a confidence reduces that to 49% while holding the residual error tolerance in 17 of 20 configurations. Adding a self-verification pass costs 2.3 times the latency of the plain configuration, reaches 44%, and is the only configuration that fails to hold the error tolerance. The practical implication is that the value of an AI workflow is set less by how often it is right than by how much of it a human must still check, and that the second property is measurable and rarely measured.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。