用自适应分治法高效检测语言事实中的全局不一致
Foundations of Global Consistency Checking with Noisy LLM Oracles
- 通过分治策略定位最小不一致子集，降低查询复杂度
- 实验表明可高效识别并定位不一致，支持实际应用
- 适合构建可靠知识库或进行事实核查的团队使用
确保自然语言事实集合的全局一致性对事实核查、摘要生成和知识库构建至关重要。尽管大型语言模型（LLMs）能评估小规模事实子集的一致性，但其判断存在噪声，成对检查不足以保证整体一致性。我们形式化该问题，证明在最坏情况下验证全局一致性需要指数级数量的预言机查询。为使任务实用，我们提出一种自适应分治算法，能够识别最小不一致子集（MUSes），并可选地通过击中集计算最小修复。该方法具有低阶多项式查询复杂度。在合成与真实LLM预言机上的实验表明，该方法能高效检测并定位不一致，为基于LLM的语义一致性验证提供可扩展框架。
原文摘要 · Abstract (English)
Ensuring that collections of natural-language facts are globally consistent is essential for tasks such as fact-checking, summarization, and knowledge base construction. While Large Language Models (LLMs) can assess the consistency of small subsets of facts, their judgments are noisy, and pairwise checks are insufficient to guarantee global coherence. We formalize this problem and show that verifying global consistency requires exponentially many oracle queries in the worst case. To make the task practical, we propose an adaptive divide-and-conquer algorithm that identifies minimal inconsistent subsets (MUSes) of facts and optionally computes minimal repairs through hitting-sets. Our approach has low-degree polynomial query complexity. Experiments with both synthetic and real LLM oracles show that our method efficiently detects and localizes inconsistencies, offering a scalable framework for linguistic consistency verification with LLM-based evaluators.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。