为大模型自托管设计企业级中间件，解决部署与集成难题
Towards a Middleware for Large Language Models
- 提出面向企业的大模型中间件架构，支持自托管服务
- 实现大模型作为应用生态入口，部分替代传统中间件功能
- 适合关注隐私、成本与定制化的中大型企业用户
大语言模型凭借处理自然语言和生成洞察的能力，接近真正的人工智能，已在全球企业服务中广泛应用。当前主要依赖OpenAI的ChatGPT和Microsoft Azure等商业云平台。随着技术成熟，企业对摆脱主流云服务商的依赖愈发强烈，推动了自托管‘大模型即服务’的需求，以满足隐私、成本与定制化要求。然而，独立部署大模型面临复杂性高、与现有系统集成困难等挑战。本文提出一种前瞻性的中间件系统架构，旨在促进大模型在企业中的部署与应用，尤其适用于大模型作为完整应用生态入口、部分承担传统中间件功能的高级场景。
原文摘要 · Abstract (English)
Large language models have gained widespread popularity for their ability to process natural language inputs and generate insights derived from their training data, nearing the qualities of true artificial intelligence. This advancement has prompted enterprises worldwide to integrate LLMs into their services. So far, this effort is dominated by commercial cloud-based solutions like OpenAI's ChatGPT and Microsoft Azure. As the technology matures, however, there is a strong incentive for independence from major cloud providers through self-hosting "LLM as a Service", driven by privacy, cost, and customization needs. In practice, hosting LLMs independently presents significant challenges due to their complexity and integration issues with existing systems. In this paper, we discuss our vision for a forward-looking middleware system architecture that facilitates the deployment and adoption of LLMs in enterprises, even for advanced use cases in which we foresee LLMs to serve as gateways to a complete application ecosystem and, to some degree, absorb functionality traditionally attributed to the middleware.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。