用分阶段提示链提升法律合同问答准确率
Answering Questions in Stages: Prompt Chaining for Contract QA
- 将复杂问题拆解为两阶段提示，逐步推理答案
- 在长且复杂的合同条款中，准确率显著优于单一提示
- 适合需要高精度的法律尽调、风险评估场景
在合同法律分析中，回答条款相关问题是关键任务，但大规模处理仍具挑战。已有方法使用简单零样本提示，对简洁条款有效，但在长篇或信息冗余的条款中表现不佳。本文提出两阶段提示链方法，用于回答多选与多选题，实验证明其在复杂法律文本上优于传统单阶段提示。我们分析了该方法的优势场景及改进空间，尤其在语言变体复杂时需增强第一阶段的问题针对性。未来工作将聚焦于提升第一阶段结果的精确匹配能力。
原文摘要 · Abstract (English)
Finding answers to legal questions about clauses in contracts is an important form of analysis in many legal workflows (e.g., understanding market trends, due diligence, risk mitigation) but more important is being able to do this at scale. Prior work showed that it is possible to use large language models with simple zero-shot prompts to generate structured answers to questions, which can later be incorporated into legal workflows. Such prompts, while effective on simple and straightforward clauses, fail to perform when the clauses are long and contain information not relevant to the question. In this paper, we propose two-stage prompt chaining to produce structured answers to multiple-choice and multiple-select questions and show that they are more effective than simple prompts on more nuanced legal text. We analyze situations where this technique works well and areas where further refinement is needed, especially when the underlying linguistic variations are more than can be captured by simply specifying possible answers. Finally, we discuss future research that seeks to refine this work by improving stage one results by making them more question-specific.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。