用法律知识图谱提升大模型推理能力，让法律问答更准确。
Knowledge Graph-Assisted LLM Post-Training for Enhanced Legal Reasoning
- 基于IRAC框架构建1.2万例案件的知识图谱，指导模型学习法律逻辑
- 在5个法律基准上4项任务表现超越基线，70B模型在6项推理任务中领先
- 适合法律AI研发者、司法科技从业者及高精度推理需求场景
大模型后训练主要依赖大规模文本和人工反馈，未能捕捉领域知识结构，导致在高风险专业领域复杂推理任务中表现不佳。法律推理尤其需要理解法律概念间的深层关系，而当前训练方法缺失这一关键要素。本文提出一种知识图谱（KG）辅助的法律大模型后训练方法，具有跨领域的可扩展性。我们基于IRAC（问题、规则、分析、结论）框架建模核心法律概念，构建包含1.2万例法律案件的知识图谱，并以此生成训练数据，对三款SOTA大模型（30B、49B、70B）分别进行监督微调（SFT）与直接偏好优化（DPO）。实验显示，后训练模型在5个多样化法律基准（共14项任务）中的平均表现优于基线，在4/6推理任务中超越基线及一款141B的SOTA法律模型，证明该知识图谱能有效增强大模型的法律推理能力。
原文摘要 · Abstract (English)
LLM post-training has primarily relied on large text corpora and human feedback, without capturing the structure of domain knowledge. This has caused models to struggle dealing with complex reasoning tasks, especially for high-stakes professional domains. In Law, reasoning requires deep understanding of the relations between various legal concepts, a key component missing in current LLM post-training. In this paper, we propose a knowledge graph (KG)-assisted approach for enhancing LLMs' reasoning capability in Legal that is generalizable to other high-stakes domains. We model key legal concepts by following the \textbf{IRAC} (Issue, Rule, Analysis and Conclusion) framework, and construct a KG with 12K legal cases. We then produce training data using our IRAC KG, and conduct both Supervised Fine-Tuning (SFT) and Direct Preference Optimization (DPO) with three state-of-the-art (SOTA) LLMs (30B, 49B and 70B), varying architecture and base model family. Our post-trained models obtained better average performance on 4/5 diverse legal benchmarks (14 tasks) than baselines. In particular, our 70B DPO model achieved the best score on 4/6 reasoning tasks, among baselines and a 141B SOTA legal LLM, demonstrating the effectiveness of our KG for enhancing LLMs' legal reasoning capability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。