从刑事判决书自动提取犯罪事实,准确率达99.5%
What Are the Facts? Automated Extraction of Court-Established Facts from Criminal-Court Opinions
- 用正则表达式和大模型双路径提取描述性内容
- 大模型方法准确率达98.75%,融合后达99.5%
- 法律专业学生评测中匹配率超90%,适合司法数据挖掘
刑事司法行政数据对犯罪行为的描述有限。然而,大陆法系国家的裁判文书包含大量关于犯罪行为的详细描述。本文研究了从斯洛伐克公开裁判文书中自动提取这些描述的可行性。采用两种方法:正则表达式与大语言模型(LLM)。基线方法使用简单正则表达式识别前后典型词汇,仅在40.5%的判决书中成功识别。进阶正则表达式方法聚焦于“sparing”及其字母分隔形式,显著提升效果。LLM方法则通过预设指令调用Gemini Flash 2.0模型进行抽取。最终,进阶正则表达式达到97%准确率,LLM达98.75%,二者结合高达99.5%。法律专业学生评估显示,两种进阶方法与人工标注一致率约90%,基线仅为34.5%;其中LLM在91.75%案例中完全匹配人工标注,结合方法达92%。
原文摘要 · Abstract (English)
Criminal justice administrative data contain only a limited amount of information about the committed offense. However, there is an unused source of extensive information in continental European courts' decisions: descriptions of criminal behaviors in verdicts by which offenders are found guilty. In this paper, we study the feasibility of extracting these descriptions from publicly available court decisions from Slovakia. We use two different approaches for retrieval: regular expressions and large language models (LLMs). Our baseline was a simple method employing regular expressions to identify typical words occurring before and after the description. The advanced regular expression approach further focused on "sparing" and its normalization (insertion of spaces between individual letters), typical for delineating the description. The LLM approach involved prompting the Gemini Flash 2.0 model to extract the descriptions using predefined instructions. Although the baseline identified descriptions in only 40.5% of verdicts, both methods significantly outperformed it, achieving 97% with advanced regular expressions and 98.75% with LLMs, and 99.5% when combined. Evaluation by law students showed that both advanced methods matched human annotations in about 90% of cases, compared to just 34.5% for the baseline. LLMs fully matched human-labeled descriptions in 91.75% of instances, and a combination of advanced regular expressions with LLMs reached 92%.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。