分析用户用GPT-4问法律问题的行为，发现超七成未提供事实信息。
LLMs & Legal Aid: Understanding Legal Needs Exhibited Through User Queries
- 用GPT-4o零样本分类，分析用户提问特征
- 70%用户未提供案件事实，65%在寻求法律信息
- 适合研究人机交互与法律科技应用者阅读
本文基于捷克专家组织Frank Bold开展的一项实验，分析用户使用GPT-4解决法律问题的互动行为。2023年5月3日至7月25日期间，共收集1,252名用户提交的3,847条查询。不同于以往聚焦大语言模型（LLMs）准确性或幻觉倾向的研究，本研究关注用户提问维度。通过GPT-4o进行零样本分类，发现：29.95%的用户提供了案件事实信息，70.05%未提供；64.93%用户寻求法律信息，35.07%寻求行动建议；28.57%用户对模型回答提出限制性要求，71.43%未设限。研究提供用户需求的定量与定性洞察，有助于深化对用户与LLM交互机制的理解。
原文摘要 · Abstract (English)
The paper presents a preliminary analysis of an experiment conducted by Frank Bold, a Czech expert group, to explore user interactions with GPT-4 for addressing legal queries. Between May 3, 2023, and July 25, 2023, 1,252 users submitted 3,847 queries. Unlike studies that primarily focus on the accuracy, factuality, or hallucination tendencies of large language models (LLMs), our analysis focuses on the user query dimension of the interaction. Using GPT-4o for zero-shot classification, we categorized queries on (1) whether users provided factual information about their issue (29.95%) or not (70.05%), (2) whether they sought legal information (64.93%) or advice on the course of action (35.07\%), and (3) whether they imposed requirements to shape or control the model's answer (28.57%) or not (71.43%). We provide both quantitative and qualitative insight into user needs and contribute to a better understanding of user engagement with LLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。