arXiv:2503.00128cs.CLcs.AI2025-03被引 1

构建首个美国上诉法院侵权案可解释判决预测数据集

AnnoCaseLaw: A Richly-Annotated Dataset For Benchmarking Explainable Legal Judgment Prediction

  • 标注471个美国上诉法院侵权案,含司法推理关键要素
  • 三类任务基准测试显示法律先例应用仍具挑战性
  • 适合法律AI、可解释性研究者使用

全球法律系统面临案件积压、司法资源有限及程序复杂化问题。人工智能在法律判决预测(LJP)领域展现潜力,即从案件事实预测法院裁决。然而现有数据集常脱离实际,缺乏高质量注释以支持法律推理与可解释性。为此,我们推出AnnoCaseLaw,首个包含471个精心标注的美国上诉法院侵权案例的数据集,每例均附有专家标注的关键司法决策成分与相关法律概念。该数据集为更贴近人类、可解释的LJP模型奠定基础。我们定义三类法律相关任务:(1)判决预测;(2)概念识别;(3)自动案例标注,并基于行业领先大语言模型建立性能基线。结果表明,LJP仍是艰巨任务,尤其法律先例的应用极具挑战。代码与数据见https://github.com/anonymouspolar1/annocaselaw。

原文摘要 · Abstract (English)

Legal systems worldwide continue to struggle with overwhelming caseloads, limited judicial resources, and growing complexities in legal proceedings. Artificial intelligence (AI) offers a promising solution, with Legal Judgment Prediction (LJP) -- the practice of predicting a court's decision from the case facts -- emerging as a key research area. However, existing datasets often formulate the task of LJP unrealistically, not reflecting its true difficulty. They also lack high-quality annotation essential for legal reasoning and explainability. To address these shortcomings, we introduce AnnoCaseLaw, a first-of-its-kind dataset of 471 meticulously annotated U.S. Appeals Court negligence cases. Each case is enriched with comprehensive, expert-labeled annotations that highlight key components of judicial decision making, along with relevant legal concepts. Our dataset lays the groundwork for more human-aligned, explainable LJP models. We define three legally relevant tasks: (1) judgment prediction; (2) concept identification; and (3) automated case annotation, and establish a performance baseline using industry-leading large language models (LLMs). Our results demonstrate that LJP remains a formidable task, with application of legal precedent proving particularly difficult. Code and data are available at https://github.com/anonymouspolar1/annocaselaw.

法律AI可解释性判决预测数据集

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。