arXiv:2509.06356cs.CLcs.AI2025-09

用参数化方法增强法律知识注入,缓解大模型上下文压力。

PL-CA: A Parametric Legal Case Augmentation Framework

  • 将法律知识编码为可训练向量,通过LoRA注入模型前馈层。
  • 在2000+条专家标注数据上保持下游任务竞争力。
  • 适合法律AI研发者与需要高效知识注入的场景。

传统RAG在司法领域虽能缓解模型知识不足与幻觉问题,但仅直接注入检索文档,受限于上下文窗口,导致上下文过长、计算开销大,干扰注意力机制并降低下游性能。现有基准多缺乏专家标注且仅关注单一任务,无法反映真实法律场景中的多任务混合特性。为此,我们提出PL-CA,一种参数化RAG(P-RAG)框架,将语料知识编码为参数向量,并通过LoRA机制融入大语言模型的前馈网络,减轻上下文压力。同时构建了一个包含超过2000个训练与测试实例的多任务法律数据集,所有样本均经专家标注与人工验证。实验表明,该方法有效降低长上下文带来的开销,同时在下游任务上表现优于传统RAG。代码与数据集附于文末。

原文摘要 · Abstract (English)

Conventional RAG is considered one of the most effective methods for addressing model knowledge insufficiency and hallucination, particularly in the judicial domain that requires high levels of knowledge rigor, logical consistency, and content integrity. However, the conventional RAG method only injects retrieved documents directly into the model's context, which severely constrains models due to their limited context windows and introduces additional computational overhead through excessively long contexts, thereby disrupting models' attention and degrading performance on downstream tasks. Moreover, many existing benchmarks lack expert annotation and focus solely on individual downstream tasks while real-world legal scenarios consist of multiple mixed legal tasks, indicating conventional benchmarks' inadequacy for reflecting models' true capabilities. To address these limitations, we propose PL-CA, which introduces a parametric RAG (P-RAG) framework to perform data augmentation on corpus knowledge and encode this legal knowledge into parametric vectors, and then integrates this parametric knowledge into the LLM's feed-forward networks (FFN) via LoRA, thereby alleviating models' context pressure. Additionally, we also construct a multi-task legal dataset comprising more than 2000 training and test instances, which are all expert-annotated and manually verified. We conduct our experiments on our dataset, and the experimental results demonstrate that our method reduces the overhead associated with excessively long contexts while maintaining competitive performance on downstream tasks compared to conventional RAG. Our code and dataset are provided in the appendix.

法律AIRAGLoRA知识注入

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。