复现图结构法律案例检索方法,验证其可靠性和泛化能力。
A Reproducibility Study of Graph-Based Legal Case Retrieval
- 构建案件与罪名的图网络,通过关系边捕捉高阶语义关联。
- 在多个数据集上复现结果,验证方法在不同场景下的稳定性。
- 使用开源大模型提升可复现性,开源全部代码与实验数据。
法律案例检索是信息检索领域的重要任务,旨在根据查询案例返回相关案例。近年来,Tang等提出CaseLink,一种基于图的法律案例检索方法,将案例和法律罪名建模为网络中的节点,通过引用关系和共享语义建立边连接,从而捕捉超越单个文档的高阶关系。尽管该方法在图结构法律信息检索中展现出潜力,但近期研究指出其结果难以复现。为此,本文对CaseLink进行复现,评估其可靠性与泛化能力:首先复现原始实验设置;其次在额外数据集上测试;进一步采用更复杂的图表示,并引入开源大语言模型替代封闭API模型以克服局限。研究结果有助于理解图方法在法律信息检索中的应用,推动领域内可复现性建设。所有实现代码与实验资源已公开共享。
原文摘要 · Abstract (English)
Legal retrieval is a widely studied area in Information Retrieval (IR) and a key task in this domain is retrieving relevant cases based on a given query case, often done by applying language models as encoders to model case similarity. Recently, Tang et al. proposed CaseLink, a novel graph-based method for legal case retrieval, which models both cases and legal charges as nodes in a network, with edges representing relationships such as references and shared semantics. This approach offers a new perspective on the task by capturing higher-order relationships of cases going beyond the stand-alone level of documents. However, while this shift in approaching legal case retrieval is a promising direction in an understudied area of graph-based legal IR, challenges in reproducing novel results have recently been highlighted, with multiple studies reporting difficulties in reproducing previous findings. Thus, in this work we reproduce CaseLink, a graph-based legal case retrieval method, to support future research in this area of IR. In particular, we aim to assess its reliability and generalizability by (i) first reproducing the original study setup and (ii) applying the approach to an additional dataset. We then build upon the original implementations by (iii) evaluating the approach's performance when using a more sophisticated graph data representation and (iv) using an open large language model (LLM) in the pipeline to address limitations that are known to result from using closed models accessed via an API. Our findings aim to improve the understanding of graph-based approaches in legal IR and contribute to improving reproducibility in the field. To achieve this, we share all our implementations and experimental artifacts with the community.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。