为大模型结构化输出实时打信任分，自动识别错误字段。
Real-Time Trustworthiness Scoring for LLM Structured Outputs and Data Extraction
- 基于置信度估计实时评估输出可信度，无需训练数据。
- 在四个数据集上对多个大模型错误检测准确率显著提升。
- 支持复杂嵌套结构，适合企业落地时人工审核优先级分配。
当前大模型的结构化输出存在间歇性错误，阻碍了企业级AI部署。我们提出CONSTRUCT，一个实时不确定性估计器，用于为大模型结构化输出打信任分。得分越低的输出越可能包含错误，可自动分配有限的人工审核资源。CONSTRUCT还能对结构化输出中的每个字段独立评分，帮助审查者快速定位错误部分。该方法适用于任意大模型（包括无logprobs的黑盒API），无需标注数据或定制模型部署，支持具有异构字段和嵌套JSON模式的复杂结构化输出。我们还构建了首个公开的大模型结构化输出基准，包含可靠真实值。在该四数据集基准上，CONSTRUCT对多种大模型（包括Gemini 3和GPT-5）的输出错误检测精度与召回率均显著优于现有技术。
原文摘要 · Abstract (English)
Structured Outputs from current LLMs exhibit sporadic errors, hindering enterprise AI deployment. We present CONSTRUCT, a real-time uncertainty estimator that scores the trustworthiness of LLM Structured Outputs. Lower-scoring outputs are more likely to contain errors, enabling automatic prioritization of limited human review bandwidth. CONSTRUCT additionally scores the trustworthiness of each field within a Structured Output, helping reviewers quickly identify which parts of the output are incorrect. Our method is suitable for any LLM (including black-box LLM APIs without logprobs), does not require labeled training data or custom model deployment, and supports complex Structured Outputs with heterogeneous fields and nested JSON schemas. We also introduce one of the first public LLM Structured Output benchmarks with reliable ground-truth values. Over this four-dataset benchmark, CONSTRUCT detects errors in outputs from various LLMs (including Gemini 3 and GPT-5) with significantly higher precision/recall than existing techniques.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。