用受控语义层防止大模型在企业分析中生成错误结果。
GROUND: Reducing Hallucinations in LLM-Based Enterprise Analytics Through Governed Semantic Definitions
- 构建受控语义层约束大模型生成,确保符合业务定义
- 在六类错误中零幻觉,且严格保护数据访问安全
- 适合需要高可靠性的企业级数据分析场景
企业在数据仓库上进行自然语言分析的需求日益增长,但实际应用受限于模型幻觉:包括错误指标、无效关联、粒度错误、数据访问不安全及解释不支持等问题。现有文本转SQL系统多依赖数据库模式或检索文档,而企业报告还需受控的业务语义——经批准的指标、维度、关联路径、过滤条件和行级安全策略。本文提出GROUND(Governed Retrieval Over Unified Normalized Definitions)框架,将大模型生成限定在受控语义层内。该框架提供经批准的定义,将用户意图绑定至合规指标与维度,并在执行前验证生成的SQL是否符合模式、指标、关联、粒度、过滤、安全及成本规则。违反时重试或放弃。在100个合成企业报告任务的基准测试中,GROUND是唯一在全部六项评估类别中无幻觉的系统;未受控系统在多个问题中违反行级安全。仅提供精确指标定义但无访问策略的语义模式仍会泄露数据,说明治理不能被指标精度替代。该结论在真实美国NHTSA车辆安全数据集上通过人工标注黄金标准验证,并在四个来自三家厂商的模型上对抗性测试中复现。GROUND强制保证过滤与行级安全,所有模型均零违规;而依赖判断的行为(如拒绝未定义指标)仍可能出错。
原文摘要 · Abstract (English)
Natural-language analytics over enterprise data warehouses is increasingly important, but production use is limited by hallucinated metrics, invalid joins, wrong grain, unsafe data access, and unsupported explanations. Existing text-to-SQL systems often ground generation in database schemas or retrieved documentation, while enterprise reporting also requires governed business semantics: approved metrics, dimensions, join paths, filters, and row-level security. This paper introduces GROUND, Governed Retrieval Over Unified Normalized Definitions, a framework that constrains LLM-generated analytics to a governed semantic layer. GROUND supplies approved definitions, binds user intent to governed metrics and dimensions, and validates generated SQL against schema, metric, join, grain, filter, security, and cost rules before execution. On violations, it retries or abstains. In a 100-question synthetic enterprise-reporting benchmark, GROUND is compared with direct schema-only text-to-SQL, schema-RAG, and semantic-only grounding under one shared model. GROUND is the only system free of measured hallucinations across all six evaluated categories, while ungoverned systems violate row-level security on many questions. A semantic-only condition with exact metric definitions but no access policy still leaks data, showing that governance cannot be replaced by metric fidelity alone. The findings are replicated on real U.S. NHTSA vehicle-safety data with independent hand-authored gold and tested on an adversarial set across four models from three providers. GROUND's enforced guarantees, especially filters and row-level security, hold with zero violations on every model, while judgment-dependent behaviors such as refusing undefined metrics remain fallible.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。