为乌兹别克语法律问答构建双模式RAG系统,兼顾云端质量与本地隐私。
Cloud and On-Premises Deployment of Uzbek Legal RAG via Targeted Retriever Fine-Tuning

- 针对云与本地两种部署场景,设计专用检索与评估框架。
- 在178个专家标注查询上,微调后开源模型性能接近闭源模型。
- 实证表明长文本法律问答微调不划算,适合低资源语言法律NLP研究者。
为乌兹别克语法律问答部署大语言模型面临通用排行榜无法反映的挑战,尤其在低资源语言和严苛运行约束下。我们构建并运营了一个检索增强(RAG)法律助手,需同时支持两种模式:一是在云服务中以每令牌成本上限最大化答案质量;二是在客户本地部署,因法律数据不可外传,只能使用开源权重模型,在有限硬件与延迟约束下运行。由于该场景无现成评估标准,我们构建了两个领域基准:一个包含178个专家标注法律查询及正确引用段落的检索基准,以及一个包含504对专家精选问答、由LLM评分且经人类判断验证的端到端基准。在两种模式下应用这些基准,发现开源与闭源模型差距小,可通过微调低成本弥补。因此我们训练了UTE-1,当前在开源模型中对乌兹别克语表现最佳的文本嵌入模型。我们还证明,通过微调缩小性能差距既不实际(因长上下文法律问答需大量算力),也无必要(因法律条文更新频繁)。这一结论得到QLoRA实验负结果的支持。我们从真实生产系统中提炼出可复用的部署指导,并公开发布评估基准、代码及微调后的嵌入模型(UTE-1)至https://metric-ai-lab.github.io/Uzbek-Legal-RAG/,以支持未来低资源法律NLP研究。
原文摘要 · Abstract (English)
Deploying large language models for legal question answering raises challenges that general-purpose leaderboards do not capture, particularly for low-resource languages and under hard operational constraints. We report on building and operating a retrieval-augmented (RAG) legal assistant for Uzbek that must run in two regimes: a managed cloud service that maximizes answer quality within a per-token cost ceiling, and an on-premises deployment for clients whose legal data may not leave their infrastructure, restricting us to open-weight models on limited local hardware under latency constraints. Because no evaluation existed for this setting, we build two domain benchmarks: a retrieval benchmark of 178 expert-annotated legal queries with gold provision spans, and an end-to-end benchmark of 504 expert-curated question--answer pairs scored by an LLM judge whose ratings we validate against human judgments and against an independent-family judge. Applying these benchmarks under each regime, we find the open-versus-proprietary gap is small and cheaply closed by fine-tuning. Therefore, we train UTE-1, which is a state-of-the-art text embedder among open models for Uzbek. We also demonstrate that closing the performance gap via fine-tuning is both impractical due to the intensive hardware demands of long-context legal Q\&A and unnecessary, given that legal acts change frequently. We support this by reporting a negative result from a QLoRA experiment. We distill practical guidance for similar deployments, drawn from a system serving real users in production. We release our benchmarks, evaluation code and the fine-tuned embedder (UTE-1) \href{https://metric-ai-lab.github.io/Uzbek-Legal-RAG/}{at this https URL} to support future work on low-resource legal NLP.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。