微软开发者应对大模型嵌入软件的挑战，探索19种新方案。
Beyond the Comfort Zone: Emerging Solutions to Overcome Challenges in Integrating LLMs into Software Products
- 通过访谈与问卷，挖掘开发者在集成大模型时的新实践方法。
- 发现19项质量保障方案，解决大模型带来的系统不确定性问题。
- 适合正在落地大模型产品的开发团队参考借鉴。
大型语言模型（LLMs）正被广泛应用于各类软件产品中，以提升用户体验，但同时也为开发者带来诸多挑战。由于大模型的独特特性，传统软件开发与评估范式难以适用，迫使开发者突破既有经验边界。本研究通过混合方法，结合26次深度访谈与332份问卷调查，分析微软多个产品团队在实践中探索的19项新兴解决方案。这些方案聚焦于质量保障环节，揭示了开发者应对大模型不可预测性、调试困难等挑战的实际策略。研究成果为更广泛地开发与评估基于大模型的产品提供了重要洞见。
原文摘要 · Abstract (English)
Large Language Models (LLMs) are increasingly embedded into software products across diverse industries, enhancing user experiences, but at the same time introducing numerous challenges for developers. Unique characteristics of LLMs force developers, who are accustomed to traditional software development and evaluation, out of their comfort zones as the LLM components shatter standard assumptions about software systems. This study explores the emerging solutions that software developers are adopting to navigate the encountered challenges. Leveraging a mixed-method research, including 26 interviews and a survey with 332 responses, the study identifies 19 emerging solutions regarding quality assurance that practitioners across several product teams at Microsoft are exploring. The findings provide valuable insights that can guide the development and evaluation of LLM-based products more broadly in the face of these challenges.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。