arXiv:2505.19310cs.SEcs.AI2025-05被引 6

用检索增强生成技术优化服务发现,降低输入长度并提升精准度。

Retrieval-Augmented Generation for Service Discovery: Chunking Strategies and Benchmarking

  • 通过分块策略预处理OpenAPI文档,减少输入令牌数。
  • 提出发现代理机制,仅加载相关端点摘要,按需获取详细规范。
  • 在SOCBench-D和RestBench上验证,端点级方法优于简单分块,精度提升显著。

集成多个(子)系统对构建先进信息系统至关重要,尤其在动态环境中,如设计阶段需集成尚未存在的服务。传统做法依赖注册表提供端点API文档,而大语言模型可基于文档自动完成系统集成(如服务组合),但受限于输入令牌长度,需精简输入内容。当前尚不清楚如何最优预处理这些API描述。本文首先分析检索增强生成(RAG)在端点发现中的应用及现有OpenAPI的分块策略,以在保持关键信息的同时减少输入令牌长度;其次提出一种发现代理(Discovery Agent),仅接收最相关端点的摘要,并按需检索详细规格;最后使用新提出的SOCBench-D基准(跨多领域通用场景)与真实世界RestBench基准,评估不同分块方式下的端点检索准确率,再测试发现代理性能。实验表明,基于端点的方法优于朴素分块,能有效降低令牌数;依赖代理显著提升精度,但可能降低召回率,揭示仍需更强推理能力。

原文摘要 · Abstract (English)

Integrating multiple (sub-)systems is essential to create advanced Information Systems. Difficulties mainly arise when integrating dynamic environments, e.g., the integration at design time of not yet existing services. This has been traditionally addressed using a registry that provides the API documentation of the endpoints. Large Language Models have shown to be capable of automatically creating system integrations (e.g., as service composition) based on this documentation but require concise input due to input oken limitations, especially regarding comprehensive API descriptions. Currently, it is unknown how best to preprocess these API descriptions. In the present work, we (i) analyze the usage of Retrieval Augmented Generation for endpoint discovery and the chunking, i.e., preprocessing, of state-of-practice OpenAPIs to reduce the input oken length while preserving the most relevant information. To further reduce the input token length for the composition prompt and improve endpoint retrieval, we propose (ii) a Discovery Agent that only receives a summary of the most relevant endpoints nd retrieves specification details on demand. We evaluate RAG for endpoint discovery using (iii) a proposed novel service discovery benchmark SOCBench-D representing a general setting across numerous domains and the real-world RestBench enchmark, first, for the different chunking possibilities and parameters measuring the endpoint retrieval accuracy. Then, we assess the Discovery Agent using the same test data set. The prototype shows how to successfully employ RAG for endpoint discovery to reduce the token count. Our experiments show that endpoint-based approaches outperform naive chunking methods for preprocessing. Relying on an agent significantly improves precision while being prone to decrease recall, disclosing the need for further reasoning capabilities.

服务发现RAGAPI分块大模型应用

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。