分析2874条开发者讨论，揭示OpenAI API使用痛点
An Empirical Study of OpenAI API Discussions on Stack Overflow
- 从Stack Overflow提取2874条讨论，分类为9类问题
- 发现提示工程、成本管理、输出不确定性是主要挑战
- 为开发者和厂商提供可操作优化建议，适合实践者参考
大型语言模型（如OpenAI的GPT系列）在自然语言处理、软件开发、教育、医疗、金融和科研等领域产生了深远影响。然而，OpenAI API带来了与传统API不同的独特挑战，包括提示工程复杂性、基于令牌的成本管理、非确定性输出以及作为黑盒运行的问题。据我们所知，此前尚无针对开发者使用OpenAI API时遇到挑战的实证研究。为填补这一空白，我们首次通过分析来自知名问答平台Stack Overflow的2,874个与OpenAI API相关的讨论，开展了一项全面的实证研究。首先考察了这些帖子的流行度和难度；随后人工将它们归类为九个相关类别，并通过主题建模分析识别出每类中的具体挑战。基于实证发现，我们最终提出了对开发者、大语言模型供应商和研究者的可行建议。
原文摘要 · Abstract (English)
The rapid advancement of large language models (LLMs), represented by OpenAI's GPT series, has significantly impacted various domains such as natural language processing, software development, education, healthcare, finance, and scientific research. However, OpenAI APIs introduce unique challenges that differ from traditional APIs, such as the complexities of prompt engineering, token-based cost management, non-deterministic outputs, and operation as black boxes. To the best of our knowledge, the challenges developers encounter when using OpenAI APIs have not been explored in previous empirical studies. To fill this gap, we conduct the first comprehensive empirical study by analyzing 2,874 OpenAI API-related discussions from the popular Q&A forum Stack Overflow. We first examine the popularity and difficulty of these posts. After manually categorizing them into nine OpenAI API-related categories, we identify specific challenges associated with each category through topic modeling analysis. Based on our empirical findings, we finally propose actionable implications for developers, LLM vendors, and researchers.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。