用逻辑规则提升大模型在保险合同中的推理准确性
Towards Robust Legal Reasoning: Harnessing Logical LLMs in Law
- 结合大模型与逻辑规则，引导模型更准确提取合同内容
- 引导式方法使覆盖判断准确率显著优于普通大模型
- 适合需要高可靠性的法律自动化场景
法律服务高度依赖文本处理。尽管大语言模型（LLMs）展现出潜力，但在法律场景中仍需更高精度、可重复性和透明度。逻辑程序通过结构化规则和事实编码法律概念，具备可靠自动化能力，但需复杂文本抽取。本文提出一种神经符号方法，融合LLMs的自然语言理解与基于逻辑的推理，以解决上述问题。以保险合同中的覆盖性问题为例，我们使用闭源与开源LLM进行了测试。对比三种方法：普通LLM、不加引导的文本编码方式，以及基于框架的引导式编码。结果显示，在引导式方法下，LLM+逻辑组合表现出显著优势，尤其在复杂合同分析中提升了推理可靠性。
原文摘要 · Abstract (English)
Legal services rely heavily on text processing. While large language models (LLMs) show promise, their application in legal contexts demands higher accuracy, repeatability, and transparency. Logic programs, by encoding legal concepts as structured rules and facts, offer reliable automation, but require sophisticated text extraction. We propose a neuro-symbolic approach that integrates LLMs' natural language understanding with logic-based reasoning to address these limitations. As a legal document case study, we applied neuro-symbolic AI to coverage-related queries in insurance contracts using both closed and open-source LLMs. While LLMs have improved in legal reasoning, they still lack the accuracy and consistency required for complex contract analysis. In our analysis, we tested three methodologies to evaluate whether a specific claim is covered under a contract: a vanilla LLM, an unguided approach that leverages LLMs to encode both the contract and the claim, and a guided approach that uses a framework for the LLM to encode the contract. We demonstrated the promising capabilities of LLM + Logic in the guided approach.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。