梳理开发者写提示词时的25类任务和51个问题，发现多数需求未被工具满足。
Understanding Prompt Programming Tasks and Questions
- 构建提示编程任务与问题的完整分类体系
- 发现16个重要问题仍无工具支持，全部任务需手动完成
- 为未来工具设计提供明确改进方向，适合提示工程研究者与开发者
提示编程（prompt programming）使开发者无需微调基础模型即可实现如文本摘要等新功能。然而，当前开发者在调整提示时面临大量未明确定义的问题。本文通过访谈16名提示程序员、观察8位开发者的修改过程，并对50名开发者进行调查，构建了包含25项任务和51个问题的分类体系，量化各项任务与问题的重要性。随后将该分类与48个研究及商业工具对比，发现所有任务均需手动完成，且51个问题中16个关键问题未被任何工具支持。研究揭示了提示编程工具在支持能力上的重大缺口，提出了未来工具设计的关键改进方向。
原文摘要 · Abstract (English)
Prompting foundation models (FMs) like large language models (LLMs) have enabled new AI-powered software features (e.g., text summarization) that previously were only possible by fine-tuning FMs. Now, developers are embedding prompts in software, known as prompt programs. The process of prompt programming requires the developer to make many changes to their prompt. Yet, the questions developers ask to update their prompt is unknown, despite the answers to these questions affecting how developers plan their changes. With the growing number of research and commercial prompt programming tools, it is unclear whether prompt programmers' needs are being adequately addressed. We address these challenges by developing a taxonomy of 25 tasks prompt programmers do and 51 questions they ask, measuring the importance of each task and question. We interview 16 prompt programmers, observe 8 developers make prompt changes, and survey 50 developers. We then compare the taxonomy with 48 research and commercial tools. We find that prompt programming is not well-supported: all tasks are done manually, and 16 of the 51 questions -- including a majority of the most important ones -- remain unanswered. Based on this, we outline important opportunities for prompt programming tools.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。