arXiv:2504.18942cs.CLcs.AI2025-04中稿 · COLM被引 2

构建法律实务全流程数据集,揭示AI与人类律师思维差异。

LawFlow: Collecting and Simulating Lawyers' Thought Processes on Business Formation Case Studies

  • 收集法学生在真实商业设立案例中的完整推理流程
  • 发现人类思维具模块化与适应性,大模型更线性且忽略后果
  • 建议AI做辅助思考而非独立决策,适合法律AI协作研发

法律从业者,尤其是初入职场者,需应对复杂高风险任务,依赖灵活、情境敏感的推理能力。当前人工智能虽有潜力支持法律工作,但现有数据集和模型仅聚焦孤立子任务,无法捕捉真实实践中端到端的决策过程。为此,我们提出LawFlow,一个基于真实商业实体设立场景、由受训法学生完成的完整法律工作流数据集。与以往关注输入输出对或线性思维链的数据集不同,LawFlow捕捉了动态、模块化、迭代式的推理过程,体现法律实践中的模糊性、修正与客户适应策略。利用该数据集,我们对比了人类与大语言模型生成的工作流,发现人类工作流更具模块化和灵活性,而大模型则更线性、冗余,且对下游影响不敏感。研究还表明,法律专业人士更倾向让AI承担辅助角色,如头脑风暴、发现盲点和提出备选方案,而非全程执行复杂流程。结果揭示了当前大模型在支持复杂法律工作流上的局限性,也为开发更具协作性、推理感知的法律AI系统提供了方向。所有数据与代码已公开于项目主页(https://minnesotanlp.github.io/LawFlow-website/)。

原文摘要 · Abstract (English)

Legal practitioners, particularly those early in their careers, face complex, high-stakes tasks that require adaptive, context-sensitive reasoning. While AI holds promise in supporting legal work, current datasets and models are narrowly focused on isolated subtasks and fail to capture the end-to-end decision-making required in real-world practice. To address this gap, we introduce LawFlow, a dataset of complete end-to-end legal workflows collected from trained law students, grounded in real-world business entity formation scenarios. Unlike prior datasets focused on input-output pairs or linear chains of thought, LawFlow captures dynamic, modular, and iterative reasoning processes that reflect the ambiguity, revision, and client-adaptive strategies of legal practice. Using LawFlow, we compare human and LLM-generated workflows, revealing systematic differences in structure, reasoning flexibility, and plan execution. Human workflows tend to be modular and adaptive, while LLM workflows are more sequential, exhaustive, and less sensitive to downstream implications. Our findings also suggest that legal professionals prefer AI to carry out supportive roles, such as brainstorming, identifying blind spots, and surfacing alternatives, rather than executing complex workflows end-to-end. Our results highlight both the current limitations of LLMs in supporting complex legal workflows and opportunities for developing more collaborative, reasoning-aware legal AI systems. All data and code are available on our project page (https://minnesotanlp.github.io/LawFlow-website/).

法律AI思维链人机协作

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。