arXiv:2604.26180cs.DBcs.AI2026-04被引 1

让大模型生成的结论自动验真,又快又准。

Evergreen: Efficient Claim Verification for Semantic Aggregates

  • 把验证任务转为可执行的语义查询,用优化技术减少大模型调用
  • 在真实数据集上实现0.94的准确率,成本降低3.1倍
  • 适合需要高可靠性的智能问答和数据分析场景

随着近期语义查询处理引擎的发展,语义聚合已成为一种基础操作,能够使用大语言模型(LLM)将关系数据简化为自然语言摘要。然而,生成的语义聚合可能包含未被底层数据支撑的断言。验证此类断言极具挑战:它们常涉及量词、分组和跨关系比较,远超LLM上下文窗口,需结合语义与符号化处理,成本高昂。本文提出Evergreen系统,将断言验证重构为可优化的语义查询处理任务,并引入溯源记录。Evergreen将每个断言编译为可执行的声明式验证查询,可在生成聚合的同一查询引擎上运行。为降低成本,系统采用验证感知优化策略,包括早期终止、相关性排序、置信度序列估计,以及通用优化技术如算子融合、相似性过滤和提示缓存。每次验证结果附带引用,标识出支持结论的最小元组集合,其语义基于一阶逻辑的半环溯源。在基于餐厅评论和客户支持数据集的生产级工作负载基准测试中,Evergreen优化配置占据完整的成本-质量帕累托前沿。使用强模型时,保持F1为0.94的同时,成本降低3.1倍;使用弱模型时,F1达0.87,优于最强外部基线(0.83),成本降低7.0倍。

原文摘要 · Abstract (English)

With recent semantic query processing engines, semantic aggregation has become a primitive operator, enabling the reduction of a relation into a natural language aggregate using an LLM. However, the resulting semantic aggregate may contain claims that are not grounded in the underlying relation. Verifying such claims is challenging: they often involve quantifiers, groupings, and comparisons over relations that far exceed LLM context windows and require a costly combination of semantic and symbolic processing. We present Evergreen, a system that recasts claim verification as a semantic query processing task with tailored optimizations and provenance capture. Evergreen compiles each claim into a declarative semantic verification query that can execute on the same query engine used to produce the aggregate. To reduce cost, Evergreen avoids unnecessary LLM calls through verification-aware optimizations, including early stopping, relevance sorting, and estimation with confidence sequences, as well as general-purpose optimizations for semantic queries, such as operator fusion, similarity filtering, and prompt caching. Each verdict is accompanied by citations that identify a minimal set of tuples justifying the result, with semantics based on semiring provenance for first-order logic. On a benchmark of production-inspired workloads over restaurant review and customer support datasets, Evergreen's optimized configurations occupy the entire cost-quality Pareto frontier. With a strong LLM, Evergreen preserves verification quality at an F1 of 0.94 while reducing cost by 3.1x relative to unoptimized verification; with a substantially weaker LLM, it surpasses the strongest external baseline's F1 (0.87 vs. 0.83) at 7.0x lower cost.

大模型验证语义查询效率优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。