无需人工标注,自动提取法律条文中的可执行规则
De Jure: Iterative LLM Self-Refinement for Structured Extraction of Regulatory Rules
- 用大模型分步解析法律文本,逐轮优化提取结果
- 三轮评估后准确率提升,金融领域达84%的下游问答偏好度
- 适用于金融、医疗、AI治理等多领域,适合合规系统开发
法律文件包含具有法律约束力的义务,但将其转化为机器可读规则仍需大量人工与专业知识。本文提出 De Jure,一个完全自动化、跨领域的结构化规则抽取管道,无需人工标注、特定领域提示或标注数据。该方法包含四个阶段:将原始文档规范化为结构化 Markdown;利用大模型进行语义分解生成规则单元;在19个维度上通过大模型作为裁判进行多标准评估;基于评分低的条目在有限再生预算内迭代修复,且上游组件优先修复。在三个涵盖金融、医疗和人工智能治理的法规语料库上,对四种模型进行评估。金融领域中,抽取质量随迭代持续提升,三轮评估后达到峰值。在医疗与AI治理领域也表现良好,支持开源与闭源模型。在下游基于检索增强生成(RAG)的合规问答测试中,基于 De Jure 提取规则的响应在单规则检索深度下被偏好73.8%,扩大检索范围后升至84.0%,证明提取精度直接提升下游应用效果。结果表明,在复杂监管领域,显式的可解释评估标准可替代人工标注,为实现以法规为基础的大模型对齐提供可扩展、可审计路径。
原文摘要 · Abstract (English)
Regulatory documents encode legally binding obligations that LLM-based systems must respect. Yet converting dense, hierarchically structured legal text into machine-readable rules remains a costly, expert-intensive process. We present De Jure, a fully automated, domain-agnostic pipeline for extracting structured regulatory rules from raw documents, requiring no human annotation, domain-specific prompting, or annotated gold data. De Jure operates through four sequential stages: normalization of source documents into structured Markdown; LLM-driven semantic decomposition into structured rule units; multi-criteria LLM-as-a-judge evaluation across 19 dimensions spanning metadata, definitions, and rule semantics; and iterative repair of low-scoring extractions within a bounded regeneration budget, where upstream components are repaired before rule units are evaluated. We evaluate De Jure across four models on three regulatory corpora spanning finance, healthcare, and AI governance. On the finance domain, De Jure yields consistent and monotonic improvement in extraction quality, reaching peak performance within three judge-guided iterations. De Jure generalizes effectively to healthcare and AI governance, maintaining high performance across both open- and closed-source models. In a downstream compliance question-answering evaluation via RAG, responses grounded in De Jure extracted rules are preferred over prior work in 73.8% of cases at single-rule retrieval depth, rising to 84.0% under broader retrieval, confirming that extraction fidelity translates directly into downstream utility. These results demonstrate that explicit, interpretable evaluation criteria can substitute for human annotation in complex regulatory domains, offering a scalable and auditable path toward regulation-grounded LLM alignment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。