用大模型自动把网页文档转为可机器读的API规范
OASBuilder: Generating OpenAPI Specifications from Online API Documentation with Large Language Models
- 结合大模型与规则算法，解析网页文档结构
- 跨数百个API生成有效OpenAPI规范，信息保留率高
- 企业级应用已节省数千小时人工，适配大模型调用
AI代理与业务自动化工具需标准化、机器可读的API规范来对接外部服务。然而，网络上的API信息多以非结构化的自由文本HTML文档形式存在，用户需耗费大量时间手动转换。为此，我们提出OASBuilder框架，将长且多样的API文档页转化为一致、机器可读的API规范。该框架通过精心设计的流水线，融合大语言模型与基于规则的算法，并依托对文档网页结构的领域知识进行引导。实验表明，OASBuilder在数百个API上具有良好泛化能力，生成的OpenAPI规范均有效，完整封装了原始文档的大部分信息。该方案已在企业环境中成功落地，节省数千小时人工工作量，使数百个复杂企业API可作为工具供大模型调用。
原文摘要 · Abstract (English)
AI agents and business automation tools interacting with external web services require standardized, machine-readable information about their APIs in the form of API specifications. However, the information about APIs available online is often presented as unstructured, free-form HTML documentation, requiring external users to spend significant time manually converting it into a structured format. To address this, we introduce OASBuilder, a novel framework that transforms long and diverse API documentation pages into consistent, machine-readable API specifications. This is achieved through a carefully crafted pipeline that integrates large language models and rule-based algorithms which are guided by domain knowledge of the structure of documentation webpages. Our experiments demonstrate that OASBuilder generalizes well across hundreds of APIs, and produces valid OpenAPI specifications that encapsulate most of the information from the original documentation. OASBuilder has been successfully implemented in an enterprise environment, saving thousands of hours of manual effort and making hundreds of complex enterprise APIs accessible as tools for LLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。