构建中文民事案件法律关系抽取基准，揭示大模型在此任务上的不足。
LexRel: Benchmarking Legal Relation Extraction for Chinese Civil Cases
- 设计层次化法律关系框架并定义论元，建立可扩展的抽取标准。
- 在LexRel基准上测试主流大模型，发现其识别准确率普遍偏低。
- 显式建模法律关系可提升其他法律AI任务性能，适合法律NLP研究者使用。
法律关系是民事案件纠纷解决的重要分析框架。然而，由于缺乏全面的分类体系，中文民事案件中的法律关系在法律人工智能领域仍处于未充分探索状态。本文首先提出一个涵盖层次化分类与论元定义的完整法律关系框架；基于该框架，构建了面向中文民事法领域的法律关系抽取任务，并发布了由专家标注的基准数据集LexRel。利用LexRel评估当前最先进的大语言模型在法律关系抽取任务上的表现，结果显示现有模型在准确识别民事法律关系方面存在显著局限性。此外，实验表明，在下游法律AI任务中显式引入法律关系信息可带来显著性能提升。
原文摘要 · Abstract (English)
Legal relations serve as an important analytical framework for dispute resolution in civil cases. However, legal relations in Chinese civil cases remain underexplored in the field of legal AI, largely due to the absence of comprehensive schemas. In this work, we first introduce a comprehensive schema for legal relations in civil cases, which contains a hierarchical taxonomy and definitions of arguments. Based on this schema, we formulate a legal relation extraction task and present LexRel, an expert-annotated benchmark for legal relation extraction in the Chinese civil law domain. We use LexRel to evaluate state-of-the-art large language models (LLMs) on legal relation extraction, showing that current LLMs exhibit significant limitations in accurately identifying civil legal relations. Furthermore, we demonstrate that explicitly incorporating information about legal relations leads to promising performance gains on other downstream legal AI tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。