开发者用自然语言提示构建智能软件,这本质上是一种新型编程方式。
Prompts Are Programs Too! Understanding How Developers Build Software Containing Prompts
- 通过与基础模型交互,开发者建立对提示行为的直观理解,而非传统代码思维。
- 20位开发者调研显示,即使经验丰富的提示工程师仍难以形成稳定认知模型。
- 提示编程与传统开发不同,适合关注AI工具链优化的研究者和工程团队。
生成式预训练模型驱动了数百万用户使用的智能软件功能,这些功能由开发者编写的自然语言提示控制。尽管提示驱动的软件影响巨大,但其开发过程与编程的关系尚不明确。本文提出,某些提示本身就是程序,而提示开发是一种独特的编程现象,称为“提示编程”。我们通过与20位在多种场景、模型、领域和提示结构中参与提示开发的开发者进行访谈,采用斯特劳斯式扎根理论,提炼出15项观察,初步揭示当前提示编程实践。例如,提示程序员不构建代码的思维模型,而是通过与基础模型(FM)互动来理解其在提示下的行为。尽管已有研究指出专家具备良好心智模型,但我们发现,即便开发过数十个提示的程序员,仍难以建立可靠的心智模型。这些发现表明提示编程不同于传统软件开发,为提示编程工具的设计提供了依据,并对软件工程相关方具有重要启示。
原文摘要 · Abstract (English)
Generative pre-trained models power intelligent software features used by millions of users controlled by developer-written natural language prompts. Despite the impact of prompt-powered software, little is known about its development process and its relationship to programming. In this work, we argue that some prompts are programs and that the development of prompts is a distinct phenomenon in programming known as "prompt programming". We develop an understanding of prompt programming using Straussian grounded theory through interviews with 20 developers engaged in prompt development across a variety of contexts, models, domains, and prompt structures. We contribute 15 observations to form a preliminary understanding of current prompt programming practices. For example, rather than building mental models of code, prompt programmers develop mental models of the foundation model (FM)'s behavior on the prompt by interacting with the FM. While prior research shows that experts have well-formed mental models, we find that prompt programmers who have developed dozens of prompts still struggle to develop reliable mental models. Our observations show that prompt programming differs from traditional software development, motivating the creation of prompt programming tools and providing implications for software engineering stakeholders.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。