arXiv:2604.06173cs.IRcs.AI2026-04中稿 · ACL

针对法规类法律问答,构建结构感知与安全评估新基准。

Beyond Case Law: Evaluating Structure-Aware Retrieval and Safety in Statute-Centric Legal QA

论文配图:Beyond Case Law: Evaluating Structure-Aware Retrieval and Safety in Statute-Centric Legal QA
图 1 · 摘自论文原文
  • 基于层级法规结构设计检索机制,提升碎片化证据召回率。
  • 模型在缺失关键条款时更易幻觉,安全拒答能力不足。
  • 适合法律AI研发者与合规系统评估人员参考。

法律问答基准长期聚焦判例法,忽视了以法规为核心的监管推理独特挑战。在法规领域,相关证据分散于层级关联的文档中,导致传统检索器失效,模型常在上下文不完整时产生幻觉。我们提出SearchFireSafety——一个面向法规类法律问答的结构感知与安全评估基准,以消防安全法规为例进行实例化。该基准通过真实问题(需引用意识检索)与合成部分上下文场景(测试幻觉与拒绝行为)双重评估模型表现。多大模型实验表明,图引导检索显著提升性能,但暴露关键安全权衡:领域适配模型在缺少关键法规证据时更易幻觉。研究强调,在法规型监管场景中,需同时评估层次化检索能力与模型安全性。

原文摘要 · Abstract (English)

Legal QA benchmarks have predominantly focused on case law, overlooking the unique challenges of statute-centric regulatory reasoning. In statutory domains, relevant evidence is distributed across hierarchically linked documents, creating a statutory retrieval gap where conventional retrievers fail and models often hallucinate under incomplete context. We introduce SearchFireSafety, a structure- and safety-aware benchmark for statute-centric legal QA. Instantiated on fire-safety regulations as a representative case, the benchmark evaluates whether models can retrieve hierarchically fragmented evidence and safely abstain when statutory context is insufficient. SearchFireSafety adopts a dual-source evaluation framework combining real-world questions that require citation-aware retrieval and synthetic partial-context scenarios that stress-test hallucination and refusal behavior. Experiments across multiple large language models show that graph-guided retrieval substantially improves performance, but also reveal a critical safety trade-off: domain-adapted models are more likely to hallucinate when key statutory evidence is missing. Our findings highlight the need for benchmarks that jointly evaluate hierarchical retrieval and model safety in statute-centric regulatory settings.

法律AI检索增强模型安全

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。