构建英国劳动法庭案件结果预测数据集，助力司法智能化
The CLC-UKET Dataset: Benchmarking Case Outcome Prediction for the UK Employment Tribunal
- 用大模型自动标注1.9万件英国劳动法庭案件
- 微调的Transformer模型预测准确率高于零样本大模型
- 适合法律AI研究者与司法科技从业者参考
本文探索技术革新与司法可及性的结合，针对英国劳动法庭（UKET）案件结果预测问题，开发首个基准数据集。为解决人工标注成本高的难题，研究采用大语言模型（LLM）实现自动化标注，构建了包含约1.9万例案件及其元数据的CLC-UKET数据集。该数据集涵盖事实、诉求、判例引用、法律条文引用、判决结果、裁判理由和管辖代码等全面法律标注。基于此数据集，开展多类案件结果预测任务，并收集人工预测作为模型性能参照。实证结果表明，微调的Transformer模型在预测任务中优于零样本和少样本大模型；通过在少样本示例中融入任务相关信息，可提升零样本大模型的表现。本研究期望所发布的数据集、人工标注与实证成果能为劳动争议解决的智能化提供重要基准。
原文摘要 · Abstract (English)
This paper explores the intersection of technological innovation and access to justice by developing a benchmark for predicting case outcomes in the UK Employment Tribunal (UKET). To address the challenge of extensive manual annotation, the study employs a large language model (LLM) for automatic annotation, resulting in the creation of the CLC-UKET dataset. The dataset consists of approximately 19,000 UKET cases and their metadata. Comprehensive legal annotations cover facts, claims, precedent references, statutory references, case outcomes, reasons and jurisdiction codes. Facilitated by the CLC-UKET data, we examine a multi-class case outcome prediction task in the UKET. Human predictions are collected to establish a performance reference for model comparison. Empirical results from baseline models indicate that finetuned transformer models outperform zero-shot and few-shot LLMs on the UKET prediction task. The performance of zero-shot LLMs can be enhanced by integrating task-related information into few-shot examples. We hope that the CLC-UKET dataset, along with human annotations and empirical findings, can serve as a valuable benchmark for employment-related dispute resolution.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。