研究开发者如何用LLM写代码，发现改提示词能显著提效率。
Understanding and supporting how developers prompt for LLM-powered code editing in practice
- 分析日志发现频繁重试提示是使用困难的信号。
- 识别出五类提示中常缺失的关键信息。
- 自动补全提示可提升27%代码修改正确率。
大型语言模型正在重塑软件工程，嵌入IDE的编程助手日益普及。尽管研究多关注工具改进和开发者感受，但对开发者实际使用流程及痛点仍缺乏理解。本文通过多阶段调查，分析谷歌内部广泛使用的IDE中名为Transform Code的LLM代码编辑功能的使用日志，发现频繁重试提示是开发者使用困难的标志。进一步对不满意的请求进行定性分析，识别出五类提示中常缺失的关键信息。基于此，提出并评估了AutoPrompter工具，通过上下文推断补全缺失信息，使测试集上的代码修改正确率提升27%。
原文摘要 · Abstract (English)
Large Language Models (LLMs) are rapidly transforming software engineering, with coding assistants embedded in an IDE becoming increasingly prevalent. While research has focused on improving the tools and understanding developer perceptions, a critical gap exists in understanding how developers actually use these tools in their daily workflows, and, crucially, where they struggle. This paper addresses part of this gap through a multi-phased investigation of developer interactions with an LLM-powered code editing feature, Transform Code, in an IDE widely used at Google. First, we analyze telemetry logs of the feature usage, revealing that frequent re-prompting can be an indicator of developer struggles with using Transform Code. Second, we conduct a qualitative analysis of unsatisfactory requests, identifying five key categories of information often missing from developer prompts. Finally, based on these findings, we propose and evaluate a tool, AutoPrompter, for automatically improving prompts by inferring missing information from the surrounding code context, leading to a 27% improvement in edit correctness on our test set.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。