针对日语法律文本检索,提出分阶段微调新方法,效果优于现有模型。
Adaptive Two-Phase Finetuning LLMs for Japanese Legal Text Retrieval
- 分两阶段微调:先学全局语境,再专攻法律复杂查询
- 在自建日语法律数据集上显著提升检索准确率
- 方法可跨语言适配,英文数据集表现也更优
文本检索(TR)旨在从大规模文档库中找到与用户查询相关的文本内容,广泛应用于法律文档检索等实际场景。尽管现有研究多集中于英语,针对日语的研究仍较匮乏。本文构建了首个专用于日语法律场景的数据集,并提出一种新型两阶段微调框架。第一阶段使模型学习全局上下文理解能力,增强对多样化查询的泛化性;第二阶段针对性地优化模型以应对法律领域的复杂查询。大量实验表明,该方法在自建日语法律数据集上显著优于现有基线。此外,该框架在英文场景下也表现优异,在MS MARCO数据集上超越同类方法。代码已公开于GitHub,模型权重可通过HuggingFace获取。
原文摘要 · Abstract (English)
Text Retrieval (TR) involves finding and retrieving text-based content relevant to a user's query from a large repository, with applications in real-world scenarios such as legal document retrieval. While most existing studies focus on English, limited work addresses Japanese contexts. In this paper, we introduce a new dataset specifically designed for Japanese legal contexts and propose a novel two-phase pipeline tailored to this domain. In the first phase, the model learns a broad understanding of global contexts, enhancing its generalization and adaptability to diverse queries. In the second phase, the model is fine-tuned to address complex queries specific to legal scenarios. Extensive experiments are conducted to demonstrate the superior performance of our method, which outperforms existing baselines. Furthermore, our pipeline proves effective in English contexts, surpassing comparable baselines on the MS MARCO dataset. We have made our code publicly available on GitHub, and the model checkpoints are accessible via HuggingFace.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。