构建首个希腊法律条文检索基准，解决法律问答中的法规匹配难题。
GreekBarRetrieval: A Benchmark for Greek Statutory Retrieval
- 基于真实律师考试题设计检索任务，融合案件事实与法律条文。
- 密集检索优于稀疏检索，经大模型重写查询后，BM25性能显著提升。
- 提出十轮类ReAct的查询重写机制，适合法律信息检索研究者使用。
法律条文检索对基于引文的法律问答至关重要，但针对希腊语的研究仍较少。本文提出GreekBarRetrieval，一个源自且补充GreekBarBench的新公开检索基准，包含283道律师考试题，每题附带案件事实，共6,308个候选法律条文。问题与事实以日常语言表述，需映射至正式法律术语与抽象概念。并非所有事实都相关。实验对比三种BM25变体与九种密集检索器发现，原始密集检索在Recall@100上远超稀疏检索。而大模型驱动的查询重写可缩小该差距，并进一步提升密集检索性能。本文提出的十轮类ReAct式重写循环使BM25在Recall@100上表现更优，且取得所有测试检索器中最高的nDCG与MAP得分。该方法优于伪相关反馈、稀疏-密集融合及英文翻译。
原文摘要 · Abstract (English)
Statutory retrieval is necessary for citation-grounded legal question answering, but remains underexplored for Greek. We introduce GreekBarRetrieval, a public retrieval benchmark derived from, and complementing GreekBarBench, which did not include retrieval. The new benchmark comprises 283 bar-exam questions, each accompanied by the facts of the case it refers to, and 6,308 candidate statutory articles to retrieve from. Questions and facts are stated in everyday language, but need to be mapped to the formal terminology of statutes and their abstract legal concepts. A further complication is that not all of the case facts are relevant to each question of a case. Experimenting with three BM25 variants and nine dense retrievers, we find that vanilla dense retrieval far outperforms vanilla sparse retrieval in Recall@100. However, LLM-based query reformulation helps BM25 close that gap, while also improving dense retrieval. With a ten-round ReAct-like LLM reformulation loop that we introduce, BM25 improves further in Recall@100 and obtains the best nDCG and MAP scores of all tested retrievers. Query reformulation also outperforms pseudo-relevance feedback, sparse-dense fusion, and English translation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。