用GPT-4和工具选择让非技术人员也能实时分析代码库数据
A Tool-Augmented, GPT-4 Chatbot for Real-Time Repository Data Analysis
- 先解析问题提取参数，再选合适工具，最后生成详细回答
- 在五类问题上准确率显著优于传统多组件系统
- 适合开发团队、管理者等缺乏查询技能的用户
软件仓库包含大量关于代码贡献、缺陷报告和项目活动的数据，但非技术利益相关者和开发者因缺乏查询能力而难以获取。为此，我们提出一种新型聊天机器人架构，利用OpenAI的GPT-4模型自动提取和分析仓库数据。该架构首先解析用户问题以提取关键参数，再根据分析结果选择合适工具，最后调用GPT-4生成详细响应。与以往依赖嵌入模型和文档检索的多组件系统不同，本方法通过提示工程和工具选择来匹配查询意图，实现流程反转。我们在包括Issues、Pull Requests、Commits、复合问题及通用仓库信息在内的多种问题类型上进行实验，评估目标提示对模型响应准确性的提升效果。结果表明，该架构能为技术与非技术人员提供可操作的洞察，显著提升仓库数据的可访问性。
原文摘要 · Abstract (English)
Software repositories contain vast amounts of data on code contributions, bug reports, and project activities, yet this information remains challenging for non-technical stakeholders and developers to access due to limited expertise in querying repositories. To address this, we introduce a novel chatbot architecture leveraging OpenAI's GPT-4 model for automated extraction and analysis of repository data. In contrast, our architecture takes a structured path first by parsing the user's query to extract relevant parameters, then selecting the correct tool to employ based on that analysis, and finally invoking the GPT-4 model to create a highly detailed response. In contrast to previous work based on multi-component systems with embedding models and document retrievers, our architecture inverts the process by relying on prompt engineering and tool selection to fit with the query intent. To validate our approach, we conducted experiments on various question types, including Issues, Pull Requests, Commits, Compound Questions, and General Repository Information, evaluating our target prompts' ability to improve the accuracy of responses from the model. Beyond demonstrating the utility of this architecture to a diverse set of users, our findings suggest that this architecture can make repository data more accessible to technical and non-technical audiences through the production of actionable insights.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。