法律AI落地三大挑战：数据难找、标注难做、结果难验。
Tasks and Roles in Legal AI: Data Curation, Annotation, and Verification
- 构建法律AI需解决文档分散、格式不一等数据难题。
- 人工标注法律条文需专家经验，现有AI辅助效率仍有限。
- 强调可验证性，呼吁跨领域合作与开源共享。
将AI应用于法律领域看似自然：大规模法律文档可用专用AI提升律师工作效率，缓解弱势群体的“司法差距”。然而，法律文本不同于支撑多数AI系统的网络文本。法律AI面临领域特异性挑战，并需在高风险场景中表现可靠。我们识别出对从业者尤为关键的三个方向：数据整理、数据标注和输出验证。首先，获取可用法律文本困难，因法律资料在技术、经济和管辖权层面存在不一致、模拟化和分散化问题。尽管AI可辅助文档整理，但现有数据匮乏也制约了其性能。其次，法律数据标注通常需要专业知识以识别复杂的司法推理模式或决定性判例。我们通过案例研究展示部分AI系统如何提升人工标注效率，同时指出其性能不足之处。最后，法律中的AI工作只有在结果可验证且可信的前提下才有价值。我们描述了AI系统支持自身输出评估的能力，以及在复杂领域进行系统性评估的新方法。我们呼吁法律与AI从业者跨学科协作，并开放共享资源，以推动新型、高性能且可靠的法律AI工具发展。
原文摘要 · Abstract (English)
The application of AI tools to the legal field feels natural: large legal document collections could be used with specialized AI to improve workflow efficiency for lawyers and ameliorate the "justice gap" for underserved clients. However, legal documents differ from the web-based text that underlies most AI systems. The challenges of legal AI are both specific to the legal domain, and confounded with the expectation of AI's high performance in high-stakes settings. We identify three areas of special relevance to practitioners: data curation, data annotation, and output verification. First, it is difficult to obtain usable legal texts. Legal collections are inconsistent, analog, and scattered for reasons technical, economic, and jurisdictional. AI tools can assist document curation efforts, but the lack of existing data also limits AI performance. Second, legal data annotation typically requires significant expertise to identify complex phenomena such as modes of judicial reasoning or controlling precedents. We describe case studies of AI systems that have been developed to improve the efficiency of human annotation in legal contexts and identify areas of underperformance. Finally, AI-supported work in the law is valuable only if results are verifiable and trustworthy. We describe both the abilities of AI systems to support evaluation of their outputs, as well as new approaches to systematic evaluation of computational systems in complex domains. We call on both legal and AI practitioners to collaborate across disciplines and to release open access materials to support the development of novel, high-performing, and reliable AI tools for legal applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。