对比13个法律专用与9个通用模型，发现法律专用模型在合同分类中更准且抗偏见。
Evaluating Customized vs. Generalist Transformer-based Models for Legal Contract Classification
- 用13个法律专用模型对比9个通用模型，评估其在3个合同分类任务中的表现。
- 法律专用模型在需要法律理解的任务上显著更优，罕见类误分类率降低。
- Legal-BERT和Contracts-BERT以更少参数刷新两项任务纪录，适合法律场景应用。
尽管法律自然语言处理取得进展，但针对合同分类任务的基于Transformer的法律专用模型（本文称'法律专用'模型）仍缺乏全面评估。为填补这一空白，我们对13个法律专用Transformer模型在3个英文合同分类任务上的表现进行了评估，并与9个通用模型进行对比。结果表明，法律专用模型在需要精细法律理解的任务中持续优于通用模型，尤其在数据分布不均时能有效减少罕见类别误分类。Legal-BERT和Contracts-BERT在三个任务中的两个上建立新SOTA，参数量仅为最佳通用模型的31%。此外，CaseLaw-BERT和LexLM也被识别为合同分类的有力基线。研究揭示了通用模型的局限性，强调在法律应用中进行领域定制的必要性。
原文摘要 · Abstract (English)
Despite advances in legal NLP, no comprehensive evaluation of Transformer-based models customized for legal tasks (referred to as `legal-specific' models in this paper) exists for contract classification tasks. To address this gap, we present an evaluation of 13 legal-specific transformer-based models on 3 English-language contract classification tasks and compare them with 9 generalist models. The results show that legal-specific models consistently outperform generalist models, especially on tasks requiring nuanced legal understanding. They also help reduce misclassification of rare classes in imbalanced datasets. Legal-BERT and Contracts-BERT establish new SOTAs on two of the three tasks, despite having 69% fewer parameters than the best-performing generalist models. We also identify CaseLaw-BERT and LexLM as strong additional baselines for contract classification. Our results highlight the shortcomings of generalist models, emphasizing the need for domain-specific customization, particularly in the context of legal applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。