用大模型理解自然语言并自动生成API测试数据，提升开发效率。
Harnessing LLMs for API Interactions: A Framework for Classification and Synthetic Data Generation
- 用大模型将自然语言转为API调用，简化软件交互
- 生成定制化数据集，验证不同模型在API分类上的表现
- 适合开发者评估大模型在特定场景下的适用性
随着大语言模型（LLMs）在自然语言处理中的进步，人们越来越关注如何利用其能力简化软件交互。本文提出一个新系统，集成大模型实现自然语言输入的API调用分类，并自动创建针对特定API功能的样本数据集。通过将自然语言指令转化为API调用，该系统使用户能以简单输入触发复杂功能，提升交互效率，降低使用门槛。生成的数据集还可用于高效、系统地评估不同大模型在API调用分类上的表现，为开发者或企业主提供实用工具，辅助其选择适合定制化API管理的大模型。我们在多个主流大模型上使用生成的样本数据集进行了实验，结果显示GPT-4分类准确率达0.996，而LLaMA-3-8B仅为0.759。这些结果凸显了大模型在重构API管理中的潜力，并验证了本系统在引导模型测试与选型方面的有效性。
原文摘要 · Abstract (English)
As Large Language Models (LLMs) advance in natural language processing, there is growing interest in leveraging their capabilities to simplify software interactions. In this paper, we propose a novel system that integrates LLMs for both classifying natural language inputs into corresponding API calls and automating the creation of sample datasets tailored to specific API functions. By classifying natural language commands, our system allows users to invoke complex software functionalities through simple inputs, improving interaction efficiency and lowering the barrier to software utilization. Our dataset generation approach also enables the efficient and systematic evaluation of different LLMs in classifying API calls, offering a practical tool for developers or business owners to assess the suitability of LLMs for customized API management. We conduct experiments on several prominent LLMs using generated sample datasets for various API functions. The results show that GPT-4 achieves a high classification accuracy of 0.996, while LLaMA-3-8B performs much worse at 0.759. These findings highlight the potential of LLMs to transform API management and validate the effectiveness of our system in guiding model testing and selection across diverse applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。