用大模型自动把混乱的API文档转成可用工具，省时省力。
ToolFactory: Automating Tool Generation by Leveraging LLM to Understand REST API Documentations
- 利用大模型解析非标准API文档，自动生成AI可用工具。
- 在167份文档、744个接口上测试，生成准确率高。
- 适合科研人员快速对接科学类API到AI系统。
基于大模型的工具代理可通过自然语言与计算服务交互。尽管REST API是构建此类代理的重要资源，但需先转换为AI兼容工具。从非结构化API文档中自动生成AI兼容工具，可大幅简化工具代理开发并降低用户学习成本。然而，API文档常缺乏标准化、模式不一致且信息不全。为此，我们开发了开源管道ToolFactory，用于从非结构化API文档中自动化生成工具。为提升生成工具的可靠性，我们设计了一种错误诊断评估方法，并构建了一个经验证工具的知识库，用于推断信息缺失的低质量文档。我们创建了包含167份文档和744个端点的API提取基准数据集，并设计了JSON标注格式。该数据集用于训练和验证ToolFactory。实验表明其有效性强。我们还通过构建糖材料研究领域的专用AI代理，展示了ToolFactory的应用潜力。ToolFactory在将科学类REST API无缝集成至AI工作流方面具有显著前景。
原文摘要 · Abstract (English)
LLM-based tool agents offer natural language interfaces, enabling users to seamlessly interact with computing services. While REST APIs are valuable resources for building such agents, they must first be transformed into AI-compatible tools. Automatically generating AI-compatible tools from REST API documents can greatly streamline tool agent development and minimize user learning curves. However, API documentation often suffers from a lack of standardization, inconsistent schemas, and incomplete information. To address these issues, we developed \textbf{ToolFactory}, an open-source pipeline for automating tool generation from unstructured API documents. To enhance the reliability of the developed tools, we implemented an evaluation method to diagnose errors. Furthermore, we built a knowledge base of verified tools, which we leveraged to infer missing information from poorly documented APIs. We developed the API Extraction Benchmark, comprising 167 API documents and 744 endpoints in various formats, and designed a JSON schema to annotate them. This annotated dataset was utilized to train and validate ToolFactory. The experimental results highlight the effectiveness of ToolFactory. We also demonstrated ToolFactory by creating a domain-specific AI agent for glycomaterials research. ToolFactory exhibits significant potential for facilitating the seamless integration of scientific REST APIs into AI workflows.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。