分析用户与大模型对话中请求表达的演变规律。
Show or Tell? Modeling the evolution of request-making in Human-LLM conversations
- 将用户请求拆分为内容、角色、上下文和通用表达四部分
- 21.1万条真实对话显示，请求从简单直接演变为带更多上下文
- 新手用语多变，经验越多越趋同，适合交互设计参考
设计以用户为中心的大模型系统需理解用户使用模式，但查询的多样性常掩盖行为规律。本文提出新框架，将用户输入分解为请求内容、角色指派、特定上下文及任务无关表达。基于WildChat数据集的21.1万条真实对话进行分析，发现人机交互中的请求语言与人际对话存在显著差异。通过历时性分析揭示：用户请求模式从初期强调单一需求，逐步融入更多上下文信息；个体虽有表达探索，但随经验积累趋于一致。据此提出应关注不同任务下共性表达趋势，并讨论其对用户研究、计算语用学及大模型对齐的启示。
原文摘要 · Abstract (English)
Designing user-centered LLM systems requires understanding how people use them, but patterns of user behavior are often masked by the variability of queries. In this work, we introduce a new framework to describe request-making that segments user input into request content, roles assigned, query-specific context, and the remaining task-independent expressions. We apply the workflow to create and analyze a dataset of 211k real-world queries based on WildChat. Compared with similar human-human setups, we find significant differences in the language for request-making in the human-LLM scenario. Further, we introduce a novel and essential perspective of diachronic analyses with user expressions, which reveals fundamental and habitual user-LLM interaction patterns beyond individual task completion. We find that query patterns evolve from early ones emphasizing sole requests to combining more context later on, and individual users explore expression patterns but tend to converge with more experience. From there, we propose to understand communal trends of expressions underlying distinct tasks and discuss the preliminary findings. Finally, we discuss the key implications for user studies, computational pragmatics, and LLM alignment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。