用大模型指令生成法提升法律文档分类准确率与效率
Improving the Accuracy and Efficiency of Legal Document Tagging with Large Language Models and Instruction Prompts
- 将多标签分类转为结构化生成任务，通过指令让大模型直接输出法律类别
- 在POSTURE50K和EURLEX57K上微调后，宏/微F1均显著优于基线模型
- 特别擅长处理标签不平衡问题，适合法律文本自动化标注场景
法律多标签分类是组织与检索海量法律文档的关键任务。尽管重要，仍面临法律语言复杂、标签依赖关系强及标签严重不均衡等挑战。本文提出Legal-LLM，利用大语言模型（LLM）的指令遵循能力，通过微调将多标签分类重构为结构化生成任务，指导LLM直接输出给定文档的相关法律类别。在两个基准数据集POSTURE50K和EURLEX57K上，以微平均和宏平均F1分数进行评估。实验结果表明，Legal-LLM显著优于多种强基线模型，包括传统方法和基于Transformer的方法。消融实验与人工评估进一步验证了该方法在处理标签不平衡及生成准确相关法律标签方面的有效性。
原文摘要 · Abstract (English)
Legal multi-label classification is a critical task for organizing and accessing the vast amount of legal documentation. Despite its importance, it faces challenges such as the complexity of legal language, intricate label dependencies, and significant label imbalance. In this paper, we propose Legal-LLM, a novel approach that leverages the instruction-following capabilities of Large Language Models (LLMs) through fine-tuning. We reframe the multi-label classification task as a structured generation problem, instructing the LLM to directly output the relevant legal categories for a given document. We evaluate our method on two benchmark datasets, POSTURE50K and EURLEX57K, using micro-F1 and macro-F1 scores. Our experimental results demonstrate that Legal-LLM outperforms a range of strong baseline models, including traditional methods and other Transformer-based approaches. Furthermore, ablation studies and human evaluations validate the effectiveness of our approach, particularly in handling label imbalance and generating relevant and accurate legal labels.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。