arXiv:2506.12895cs.CLcs.IR2025-06被引 5

对比词法与语义模型在法律文本检索中的表现,发现重复性语言下词法更优。

Assessing the Performance Gap Between Lexical and Semantic Models for Information Retrieval With Formulaic Legal Language

  • 比较BM25(词法)与稠密检索模型在欧盟法院判决文本中的表现
  • 重复性强时两类模型都有效,但长查询中词法模型更胜一筹
  • 领域微调可提升语义模型性能,适合法律检索系统优化

法律段落检索对辅助法律从业者快速查找判例至关重要。本文研究欧盟法院(CJEU)判决文本的检索任务,其语言高度结构化且模式化,常出现重复表达。通过三种互补指标的定量与定性分析发现,在语言重复性高的场景中,词法模型(如BM25)与稠密检索模型均表现良好;而在重复性较低、长查询或引用较多的细微情境中,BM25优于通用稠密模型。实验显示,BM25在7个指标中有4个超越现成稠密模型。然而,使用领域数据微调后,稠密模型在多数指标上反超BM25。同时分析了微调数据量对模型性能与时间鲁棒性的影响。代码、数据集及附录见:https://github.com/larimo/lexsem-legal-ir。

原文摘要 · Abstract (English)

Legal passage retrieval is an important task that assists legal practitioners in the time-intensive process of finding relevant precedents to support legal arguments. This study investigates the task of retrieving legal passages or paragraphs from decisions of the Court of Justice of the European Union (CJEU), whose language is highly structured and formulaic, leading to repetitive patterns. Understanding when lexical or semantic models are more effective at handling the repetitive nature of legal language is key to developing retrieval systems that are more accurate, efficient, and transparent for specific legal domains. To this end, we explore when this routinized legal language is better suited for retrieval using methods that rely on lexical and statistical features, such as BM25, or dense retrieval models trained to capture semantic and contextual information. A qualitative and quantitative analysis with three complementary metrics shows that both lexical and dense models perform well in scenarios with more repetitive usage of language, whereas BM25 performs better than the dense models in more nuanced scenarios where repetition and verbatim~quotes are less prevalent and in longer queries. Our experiments also show that BM25 is a strong baseline, surpassing off-the-shelf dense models in 4 out of 7 performance metrics. However, fine-tuning a dense model on domain-specific data led to improved performance, surpassing BM25 in most metrics, and we analyze the effect of the amount of data used in fine-tuning on the model's performance and temporal robustness. The code, dataset and appendix related to this work are available on: https://github.com/larimo/lexsem-legal-ir.

信息检索法律AI词法模型稠密检索

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。