通过结构化法律文本提升大模型推理能力
Structured Definitions and Segmentations for Legal Reasoning in LLMs: A Study on Indian Legal Data
- 按法律文本的修辞角色重新组织内容，改善长文本处理
- 解释关键法律术语使模型性能提升1.5%至4.36%（F1）
- 适合法律AI研究者和需要高精度法律推理的应用
大型语言模型（LLMs）在广泛网络数据上训练，具备出色的通用推理能力。然而，在法律等专业领域表现不佳，主要因缺乏领域特定预训练。法律文件通常冗长复杂，模型难以高效处理全文。以往研究尝试采用上下文学习方法弥补知识缺口，在无需完整领域对齐的情况下提升模型在新领域的表现。本文在三个印度法律判决预测数据集上，零样本设置下开展三项实验：(i) 根据修辞角色重组文档，评估结构化信息对长上下文处理与决策的影响；(ii) 定义修辞角色，帮助模型熟悉法律术语；(iii) 模拟法院处理修辞角色的分步推理过程，以增强模型推理能力。结果表明，结构化数据或术语定义可显著提升模型表现，F1分数最低提升约1.5%，最高达4.36%。
原文摘要 · Abstract (English)
Large Language Models (LLMs), trained on extensive datasets from the web, exhibit remarkable general reasoning skills. Despite this, they often struggle in specialized areas like law, mainly because they lack domain-specific pretraining. The legal field presents unique challenges, as legal documents are generally long and intricate, making it hard for models to process the full text efficiently. Previous studies have examined in-context approaches to address the knowledge gap, boosting model performance in new domains without full domain alignment. In our paper, we analyze model behavior on legal tasks by conducting experiments in three areas: (i) reorganizing documents based on rhetorical roles to assess how structured information affects long context processing and model decisions, (ii) defining rhetorical roles to familiarize the model with legal terminology, and (iii) emulating the step-by-step reasoning of courts regarding rhetorical roles to enhance model reasoning. These experiments are conducted in a zero-shot setting across three Indian legal judgment prediction datasets. Our results reveal that organizing data or explaining key legal terms significantly boosts model performance, with a minimum increase of ~1.5% and a maximum improvement of 4.36% in F1 score compared to the baseline.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。