arXiv:2512.08083cs.IR2025-12被引 1

利用大模型随机性提升法律文件分类准确率,增强合规决策信心。

Exploiting the Randomness of Large Language Models (LLM) in Text Classification Tasks: Locating Privileged Documents in Legal Matters

  • 通过控制大模型随机性参数,探索其对法律文书分类的影响。
  • 随机性调控对整体性能影响小,但特定方法可显著提升准确率。
  • 适合法律合规、企业风控等需要高可信度判断的场景。

在法律事务中,文本分类模型常用于从海量文档中筛选符合特定标准的文件,如律师-客户特权沟通内容和律师指示文件。大语言模型在此类任务中表现出色。本文通过实证研究,探讨大模型在律师-客户特权文件检测中的随机性作用,重点关注四个维度:(1) 大模型识别特权文件的有效性;(2) 随机性控制参数对分类输出的影响;(3) 其对整体分类性能的影响;(4) 利用随机性提升准确率的方法。实验表明,大模型能有效识别特权文件,随机性控制参数对性能影响较小,而所提出的随机性利用方法可显著提升准确率。值得注意的是,该方法可增强企业在制裁合规流程中对大模型输出的信心。随着组织越来越多依赖大模型辅助合规流程,降低输出波动有助于建立内部及监管方对大模型生成的制裁筛查决策的信任。

原文摘要 · Abstract (English)

In legal matters, text classification models are most often used to filter through large datasets in search of documents that meet certain pre-selected criteria like relevance to a certain subject matter, such as legally privileged communications and attorney-directed documents. In this context, large language models have demonstrated strong performance. This paper presents an empirical study investigating the role of randomness in LLM-based classification for attorney-client privileged document detection, focusing on four key dimensions: (1) the effectiveness of LLMs in identifying legally privileged documents, (2) the influence of randomness control parameters on classification outputs, (3) their impact on overall classification performance, and (4) a methodology for leveraging randomness to enhance accuracy. Experimental results showed that LLMs can identify privileged documents effectively, randomness control parameters have minimal impact on classification performance, and importantly, our developed methodology for leveraging randomness can have a significant impact on improving accuracy. Notably, this methodology that leverages randomness could also enhance a corporation's confidence in an LLM's output when incorporated into its sanctions-compliance processes. As organizations increasingly rely on LLMs to augment compliance workflows, reducing output variability helps build internal and regulatory confidence in LLM-derived sanctions-screening decisions.

法律AI大模型分类合规

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。