提升法律大模型可靠性，通过元数据增强检索与偏好优化。
Enhancing Legal LLMs through Metadata-Enriched RAG Pipelines and Direct Preference Optimization
- 用元数据增强混合检索，解决法律文本重复带来的召回问题。
- 结合直接偏好优化，让模型在信息不足时拒绝回答。
- 适合关注法律AI安全与隐私保护的开发者与研究者。
大型语言模型在短文本上表现良好，但在长篇法律文档中性能下降，常出现错误条款或判例等幻觉。在要求高精度的法律领域，此类错误会削弱模型的可信度。尽管检索增强生成（RAG）能提高输出的准确性，但在法律场景中仍受限，尤其是对小型本地部署模型而言。我们识别出两个关键失效模式：因法律语料库中词汇冗余导致的检索错误，以及在上下文不足时仍强行生成答案的解码错误。为此，我们提出元数据增强的混合检索（Metadata Enriched Hybrid RAG）以改善文档级检索，并应用直接偏好优化（DPO）来强制模型在上下文不充分时拒绝回答。两者结合显著提升了法律大模型的对齐性、可靠性和安全性。
原文摘要 · Abstract (English)
Large Language Models (LLMs) perform well in short contexts but degrade on long legal documents, often producing hallucinations such as incorrect clauses or precedents. In the legal domain, where precision is critical, such errors undermine reliability and trust. Retrieval Augmented Generation (RAG) helps ground outputs but remains limited in legal settings, especially with small, locally deployed models required for data privacy. We identify two failure modes: retrieval errors due to lexical redundancy in legal corpora, and decoding errors where models generate answers despite insufficient context. To address this, we propose Metadata Enriched Hybrid RAG to improve document level retrieval, and apply Direct Preference Optimization (DPO) to enforce safe refusal when context is inadequate. Together, these methods improve grounding, reliability, and safety in legal language models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。