法官个人倾向影响判案结果,机器学习证明不同法官有独特裁决模式。
The Judge Variable: Challenging Judge-Agnostic Legal Judgment Prediction
- 用大模型提取特征,分法官训练专用预测模型
- 专用模型最高准确率92.85%,远超通用模型的82.63%
- 实证支持法律现实主义,适合研究司法个体差异者
本研究通过机器学习预测法国上诉法院儿童监护权判决结果,检验法官个体决策模式是否显著影响案件走向,挑战法官中立、统一适用法律的传统假设。为遵守法国隐私法规,所有数据均经严格匿名化处理。分析基于10,306个案件中的18,937份生活安排裁决。比较基于单个法官历史裁决训练的专用模型与基于聚合数据训练的通用模型。预测流程结合大语言模型(LLMs)进行结构化特征提取和机器学习模型(随机森林、XGBoost、SVC)进行结果预测。结果显示,专用模型在预测准确性上持续优于通用模型,最佳模型F1得分达92.85%,而通用模型虽使用20至100倍更多样本,仅得82.63%。专用模型捕捉到可复现的个体判案模式,且不具备跨法官迁移性。域内与跨域验证均支持法律现实主义观点,表明法官身份对判决结果具有可测量影响。所有数据与代码将公开。
原文摘要 · Abstract (English)
This study examines the role of human judges in legal decision-making by using machine learning to predict child physical custody outcomes in French appellate courts. Building on the legal realism-formalism debate, we test whether individual judges' decision-making patterns significantly influence case outcomes, challenging the assumption that judges are neutral variables that apply the law uniformly. To ensure compliance with French privacy laws, we implement a strict pseudonymization process. Our analysis uses 18,937 living arrangements rulings extracted from 10,306 cases. We compare models trained on individual judges' past rulings (specialist models) with a judge-agnostic model trained on aggregated data (generalist models). The prediction pipeline is a hybrid approach combining large language models (LLMs) for structured feature extraction and ML models for outcome prediction (RF, XGB and SVC). Our results show that specialist models consistently achieve higher predictive accuracy than the general model, with top-performing models reaching F1 scores as high as 92.85%, compared to the generalist model's 82.63% trained on 20x to 100x more samples. Specialist models capture stable individual patterns that are not transferable to other judges. In-Domain and Cross-Domain validity tests provide empirical support for legal realism, demonstrating that judicial identity plays a measurable role in legal outcomes. All data and code used will be made available.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。