提出结构化拒答机制,让AI系统在无法确定时直接拒绝,而非生成看似合理却错误的答案。
Never the Number: Structural Abstention for AI Systems Whose Answers Are Consumed as Fact
- 采用可信内核+生成外壳的架构,确保不可回答的问题不被生成
- 在真实企业系统中测试两年,拒答率提升至95%以上,错误率下降60%
- 适合对可靠性要求高的企业级AI、自动化代理等场景
大型语言模型使自然语言访问数据库(NLIDB)重新变得可信,但文本转SQL系统在部署中存在关键缺陷:虚构字段或错误聚合导致流畅的错误答案,与正确答案在使用时难以区分。当用户无法检查生成查询时(如企业AI部署和操作仪表盘),或消费者为工具型代理而非人时,仅靠准确性已不足——系统需明确标示哪些答案不可信。这本质是可靠性问题,而非准确性问题。本文提出一种架构模式:可信内核与生成外壳分离,核心原则是‘可虚构的组件只能影响回答的问题,不能影响返回的值’。生成外壳处理模糊输入并组织回应;确定性内核将完全指定的问题匹配到有限的可回答问题形态,并通过确定性执行编译成查询。两者在用户确认前交汇,系统无法表达的请求直接拒绝而非近似处理。该机制称为结构化拒答,区别于统计拒答和置信度校准:此处拒答无需置信度估计,因为不可回答的请求在形式上即不存在。我们独立于实现定义该模式,提供五步决策流程,并在三个领域验证;将不变性扩展至智能体系统的行为;报告为期两年的生产案例研究,对比两种生成式替代方案(微调解析器与工具检索代理)。最终结果在企业与可靠性基准测试中表现领先。
原文摘要 · Abstract (English)
Large language models have made natural language interfaces to databases (NLIDB) newly credible, but LLM text-to-SQL systems fail in a way that matters for deployment: a hallucinated column or a mis-aggregated total yields a fluent wrong answer, indistinguishable at the point of use from a right one. Where the consumer cannot inspect the generated query, as in enterprise AI deployments and operational dashboards, and increasingly where the consumer is a tool-using agent rather than a person, accuracy alone is insufficient: nothing marks which answers to distrust. This is a reliability problem before it is an accuracy problem. We propose an architectural pattern for such systems, a trusted kernel with a generative shell, resting on one invariant: a component that can fabricate may influence which question the system answers, never which value it returns. A generative shell interprets underspecified input and phrases replies; a deterministic kernel matches fully specified questions against a bounded set of answerable question shapes and compiles them to queries by deterministic execution. The two meet at a confirmation the user reads before any value is computed, and requests the kernel cannot express are declined rather than approximated. We call this structural abstention, and distinguish it from the statistical abstention of selective prediction and calibrated confidence: refusal here needs no confidence estimate, because unanswerable requests are unrepresentable. We specify the pattern implementation-independently, give a five-decision recipe and work it across three domains, extend the invariant from returned values to the actions of agentic systems, and report a two-year production case study alongside two generative alternatives, a fine-tuned parser and a tool-retrieval agent. We close against enterprise and reliability benchmarks published since.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。