用知识图谱对齐检测法律AI幻觉，确保引用内容真实可追溯。
HalluGraph: Auditable Hallucination Detection for Legal RAG Systems via Knowledge Graph Alignment
- 通过构建源文档、查询与生成文本的知识图谱并比对结构，量化幻觉程度。
- 在结构化法律文件上达到AUC 0.979，生成任务中AUC约0.89，优于传统方法。
- 适合需要审计追踪的高风险法律AI场景，结果可解释、可溯源。
基于检索增强生成（RAG）的法律AI系统面临问责难题：当AI引用判例、法规或合同条款时，使用者需可信保障其生成内容忠实于原始文档。现有幻觉检测依赖语义相似性，容忍实体替换，而混淆当事人、日期或法律条文可能带来严重后果。我们提出HalluGraph，一种基于图论的框架，通过对比上下文、查询与响应提取的知识图谱结构来量化幻觉。该方法生成可界定、可解释的指标，分为实体锚定（EG）——衡量响应中的实体是否出现在源文档中，以及关系保持（RP）——验证声明的关系是否由上下文支持。在结构化控制文档上，HalluGraph实现近乎完美的区分能力（>400词，>20实体），取得AUC=0.979；在复杂生成任务中仍保持稳健性能（AUC≈0.89），持续优于语义相似性基线。该框架为高风险法律应用提供透明与可追溯性，支持从生成断言回溯至原始段落的完整审计链。
原文摘要 · Abstract (English)
Legal AI systems powered by retrieval-augmented generation (RAG) face a critical accountability challenge: when an AI assistant cites case law, statutes, or contractual clauses, practitioners need verifiable guarantees that generated text faithfully represents source documents. Existing hallucination detectors rely on semantic similarity metrics that tolerate entity substitutions, a dangerous failure mode when confusing parties, dates, or legal provisions can have material consequences. We introduce HalluGraph, a graph-theoretic framework that quantifies hallucinations through structural alignment between knowledge graphs extracted from context, query, and response. Our approach produces bounded, interpretable metrics decomposed into \textit{Entity Grounding} (EG), measuring whether entities in the response appear in source documents, and \textit{Relation Preservation} (RP), verifying that asserted relationships are supported by context. On structured control documents, HalluGraph achieves near-perfect discrimination ($>$400 words, $>$20 entities), HalluGraph achieves $AUC = 0.979$, while maintaining robust performance ($AUC \approx 0.89$) on challenging generative legal task, consistently outperforming semantic similarity baselines. The framework provides the transparency and traceability required for high-stakes legal applications, enabling full audit trails from generated assertions back to source passages.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。