构建首个覆盖多司法管辖区的法律信息检索基准
The Massive Legal Embedding Benchmark (MLEB)
- 整合10个专家标注数据集，覆盖6国法律文本
- 包含案例、法规、合同等5类文档与3类任务
- 开源代码数据，支持可复现评估
我们提出大规模法律嵌入基准（MLEB），目前最全面、最多样、最大的开源法律信息检索基准。MLEB包含10个专家标注的数据集，覆盖美国、英国、欧盟、澳大利亚、爱尔兰和新加坡等多个司法管辖区，涵盖案例、法规、监管指南、合同和文献等多种文档类型，以及搜索、零样本分类和问答等多种任务类型。其中7个数据集为新构建，以填补开源法律信息检索领域的领域与司法管辖区空白。本文详细说明了MLEB的构建方法及新数据集创建过程，并公开发布代码、结果和数据，以支持可复现的评估。
原文摘要 · Abstract (English)
We present the Massive Legal Embedding Benchmark (MLEB), the largest, most diverse, and most comprehensive open-source benchmark for legal information retrieval to date. MLEB consists of ten expert-annotated datasets spanning multiple jurisdictions (the US, UK, EU, Australia, Ireland, and Singapore), document types (cases, legislation, regulatory guidance, contracts, and literature), and task types (search, zero-shot classification, and question answering). Seven of the datasets in MLEB were newly constructed in order to fill domain and jurisdictional gaps in the open-source legal information retrieval landscape. We document our methodology in building MLEB and creating the new constituent datasets, and release our code, results, and data openly to assist with reproducible evaluations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。