用AI重写用户低效提问,让大模型回答更准确。
Conversational User-AI Intervention: A Study on Prompt Rewriting for Improved LLM Response Generation
- 用大模型自动优化用户提问措辞,提升回复质量。
- 长对话中重写效果更好,上下文有助于理解意图。
- 适合希望提升人机交互效率的开发者和研究者。
人类与大语言模型的对话日益普及,但许多用户仍难以获得有用回复。原因之一是用户缺乏有效提示词设计能力。本文首次以大模型为中心,研究真实的人机对话数据,分析用户提问中表达信息需求不充分的问题,并探索大模型重写低效提示的潜力。结果表明,重写无效提示可显著提升对话系统响应质量,同时保持用户原始意图不变。尤其在较长对话中,基于上下文推断用户需求更准确,重写效果更佳。我们还发现,大模型在解析提示时往往需做出合理假设来理解用户意图。这些发现跨不同对话领域、用户目标和模型规模均成立,说明提示重写是改善人机交互的可行方案。
原文摘要 · Abstract (English)
Human-LLM conversations are increasingly becoming more pervasive in peoples' professional and personal lives, yet many users still struggle to elicit helpful responses from LLM Chatbots. One of the reasons for this issue is users' lack of understanding in crafting effective prompts that accurately convey their information needs. Meanwhile, the existence of real-world conversational datasets on the one hand, and the text understanding faculties of LLMs on the other, present a unique opportunity to study this problem, and its potential solutions at scale. Thus, in this paper we present the first LLM-centric study of real human-AI chatbot conversations, focused on investigating aspects in which user queries fall short of expressing information needs, and the potential of using LLMs to rewrite suboptimal user prompts. Our findings demonstrate that rephrasing ineffective prompts can elicit better responses from a conversational system, while preserving the user's original intent. Notably, the performance of rewrites improves in longer conversations, where contextual inferences about user needs can be made more accurately. Additionally, we observe that LLMs often need to -- and inherently do -- make \emph{plausible} assumptions about a user's intentions and goals when interpreting prompts. Our findings largely hold true across conversational domains, user intents, and LLMs of varying sizes and families, indicating the promise of using prompt rewriting as a solution for better human-AI interactions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。