分析用户与大模型写作协作的典型行为模式,揭示互动规律。
Prototypical Human-AI Collaboration Behaviors from LLM-Assisted Writing in the Wild
- 识别出用户在提示中常见的六类协作行为模式
- 发现用户意图显著影响其与模型的互动方式
- 为大模型对齐提供可解释的行为依据
随着大语言模型(LLMs)被广泛应用于复杂写作任务,用户通过多轮交互主动调整生成内容以满足需求。我们对两位主流AI助手Bing Copilot和WildChat的真实用户写作会话进行了大规模分析。研究突破以往仅做任务分类或满意度评估的局限,系统刻画了用户在整个会话中与模型的互动特征。我们提炼出一组具有代表性的‘人类-人工智能协作行为’(PATHs),包括修正意图、探索文本、提问、调整风格或注入新内容等。这些路径能解释大多数用户交互行为的差异。进一步发现特定写作意图与特定路径存在显著相关性,表明用户目标直接影响协作策略。研究结果对提升大模型对齐能力具有重要意义。
原文摘要 · Abstract (English)
As large language models (LLMs) are used in complex writing workflows, users engage in multi-turn interactions to steer generations to better fit their needs. Rather than passively accepting output, users actively refine, explore, and co-construct text. We conduct a large-scale analysis of this collaborative behavior for users engaged in writing tasks in the wild with two popular AI assistants, Bing Copilot and WildChat. Our analysis goes beyond simple task classification or satisfaction estimation common in prior work and instead characterizes how users interact with LLMs through the course of a session. We identify prototypical behaviors in how users interact with LLMs in prompts following their original request. We refer to these as Prototypical Human-AI Collaboration Behaviors (PATHs) and find that a small group of PATHs explain a majority of the variation seen in user-LLM interaction. These PATHs span users revising intents, exploring texts, posing questions, adjusting style or injecting new content. Next, we find statistically significant correlations between specific writing intents and PATHs, revealing how users' intents shape their collaboration behaviors. We conclude by discussing the implications of our findings on LLM alignment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。