用生成案例+模拟陪审团,让大模型更懂法律合规。
AUTOLAW: Enhancing Legal Compliance in Large Language Models via Case Law Generation and Jury-Inspired Deliberation
- 生成本地化案例,动态适配不同法律区域。
- 陪审团投票机制使违规检测率显著提升。
- 适合法律、合规等高风险场景的模型评估。
领域专用大语言模型在法律等领域的快速发展,亟需能应对复杂地域法律差异的框架,以确保合规与可信。现有法律评估基准普遍缺乏灵活性,难以适应动态变化的监管环境。为此,我们提出AutoLaw,一种结合对抗性数据生成与模拟陪审团决策的违规检测框架。该框架动态生成反映本地法规的判例,并利用基于LLM的“陪审员”池模拟司法裁决过程。陪审员根据合成的法律专业度排序与选取,通过集体审议减少偏差,提升检测精度。在Law-SG、Case-SG(合法性)和Unfair-TOS(政策)三个基准上的评估表明:对抗性数据生成增强了模型对违规行为的区分能力,陪审团投票策略显著提高了违规检测率。结果表明,AutoLaw具备自适应探测法律错位的能力,可提供可靠、上下文感知的判断,为法律敏感应用中的大模型评估与优化提供可扩展解决方案。
原文摘要 · Abstract (English)
The rapid advancement of domain-specific large language models (LLMs) in fields like law necessitates frameworks that account for nuanced regional legal distinctions, which are critical for ensuring compliance and trustworthiness. Existing legal evaluation benchmarks often lack adaptability and fail to address diverse local contexts, limiting their utility in dynamically evolving regulatory landscapes. To address these gaps, we propose AutoLaw, a novel violation detection framework that combines adversarial data generation with a jury-inspired deliberation process to enhance legal compliance of LLMs. Unlike static approaches, AutoLaw dynamically synthesizes case law to reflect local regulations and employs a pool of LLM-based "jurors" to simulate judicial decision-making. Jurors are ranked and selected based on synthesized legal expertise, enabling a deliberation process that minimizes bias and improves detection accuracy. Evaluations across three benchmarks: Law-SG, Case-SG (legality), and Unfair-TOS (policy), demonstrate AutoLaw's effectiveness: adversarial data generation improves LLM discrimination, while the jury-based voting strategy significantly boosts violation detection rates. Our results highlight the framework's ability to adaptively probe legal misalignments and deliver reliable, context-aware judgments, offering a scalable solution for evaluating and enhancing LLMs in legally sensitive applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。