用确定性模型+模糊分级，让法律合规审查更透明可复现。
Deterministic Fuzzy Triage for Legal Compliance Classification and Evidence Retrieval
- 基于罗伯特双编码器与余弦相似度，实现可复现的合同条款检索
- 在测试集上达到NDCG@5 0.38-0.42，二分类任务AUC达0.98-0.99
- 通过可调阈值划分自动不合规、自动合规和需人工审查三类结果
法律团队越来越多地使用机器学习处理大量合同证据，但许多模型存在不透明、非确定性问题，难以与HIPAA或NERC-CIP等框架对齐。本文提出一种简单、可复现的方法：采用确定性双编码器与透明的模糊分级带。在ACORD基准上训练一个RoBERTa-base双编码器（512维投影，余弦相似度），用于分级条款检索；随后在基于CUAD的二分类合规数据集上微调。在单张NVIDIA A100 GPU上，五组随机种子（40-44）下，测试集上获得NDCG@5 0.38-0.42，NDCG@10 0.45-0.50，4星精确率@5约0.37。在高度不平衡的二分类任务中（正例占比约0.6%），模型达到AUC 0.98-0.99，F1 0.22-0.30，优于多数基线。进一步将合规评分映射为三类：自动不合规、自动合规、人工审核。通过验证集调优阈值，在保证自动决策错误率不超过2%的前提下最大化覆盖范围。系统最终由少量标量参数定义，具有种子稳定性。我们主张，确定性编码器、校准的模糊区间与显式误差约束，可在规则引擎与黑箱大模型之间提供实用折中方案，支持可解释的证据筛选、可复现的审计轨迹，并与法律审查概念建立明确映射。
原文摘要 · Abstract (English)
Legal teams increasingly use machine learning to triage large volumes of contractual evidence, but many models are opaque, non-deterministic, and difficult to align with frameworks such as HIPAA or NERC-CIP. We study a simple, reproducible alternative based on deterministic dual encoders and transparent fuzzy triage bands. We train a RoBERTa-base dual encoder with a 512-dimensional projection and cosine similarity on the ACORD benchmark for graded clause retrieval, then fine-tune it on a CUAD-derived binary compliance dataset. Across five random seeds (40-44) on a single NVIDIA A100 GPU, the model achieves ACORD-style retrieval performance of NDCG@5 0.38-0.42, NDCG@10 0.45-0.50, and 4-star Precision@5 about 0.37 on the test split. On CUAD-derived binary labels, it achieves AUC 0.98-0.99 and F1 0.22-0.30 depending on positive-class weighting, outperforming majority and random baselines in a highly imbalanced setting with a positive rate of about 0.6%. We then map scalar compliance scores into three regions: auto-noncompliant, auto-compliant, and human-review. Thresholds are tuned on validation data to maximize automatic decision coverage subject to an empirical error-rate constraint of at most 2% over auto-decided examples. The result is a seed-stable system summarized by a small number of scalar parameters. We argue that deterministic encoders, calibrated fuzzy bands, and explicit error constraints provide a practical middle ground between hand-crafted rules and opaque large language models, supporting explainable evidence triage, reproducible audit trails, and concrete mappings to legal review concepts.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。