评测大模型在中英文法律案例中的表现，揭示其法律推理优劣
Legal Evalutions and Challenges of Large Language Models
- 以OpenAI o1为例，对比开源、闭源及法律专用模型的法律应用能力
- 在英美法系与中国案例上测试，发现模型对法律语言理解存在偏差
- 适合法律AI研究者与司法科技从业者参考
本文基于大语言模型（LLMs）开展法律测试，以OpenAI o1模型为案例，评估大模型在适用法律条文方面的表现。对比了当前最先进的开源、闭源及专为法律领域训练的模型，在英美法系与中国法律案例上进行系统性测试，并深入分析结果。研究探讨了大模型在理解与应用法律文本、法律问题推理及判决预测方面的优势与不足，尤其指出法律语言解读与法律推理准确性方面的挑战。最后，全面分析各类模型的优劣，为人工智能在法律领域的未来应用提供重要参考。
原文摘要 · Abstract (English)
In this paper, we review legal testing methods based on Large Language Models (LLMs), using the OPENAI o1 model as a case study to evaluate the performance of large models in applying legal provisions. We compare current state-of-the-art LLMs, including open-source, closed-source, and legal-specific models trained specifically for the legal domain. Systematic tests are conducted on English and Chinese legal cases, and the results are analyzed in depth. Through systematic testing of legal cases from common law systems and China, this paper explores the strengths and weaknesses of LLMs in understanding and applying legal texts, reasoning through legal issues, and predicting judgments. The experimental results highlight both the potential and limitations of LLMs in legal applications, particularly in terms of challenges related to the interpretation of legal language and the accuracy of legal reasoning. Finally, the paper provides a comprehensive analysis of the advantages and disadvantages of various types of models, offering valuable insights and references for the future application of AI in the legal field.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。