综述大模型驱动智能代理服务的部署技术，助力实现高可靠、可扩展的通用人工智能应用。
Deploying Foundation Model Powered Agent Services: A Survey
- 提出统一框架，融合模型与资源优化，支撑跨异构设备部署。
- 涵盖推理加速、并行计算与资源动态调度等关键技术。
- 适合关注AI服务部署、系统架构与大模型工程化的研究者。
大模型（Foundation Model, FM）驱动的智能代理服务被视为迈向通用人工智能（AGI）的重要路径。为实现高可靠性与可扩展性部署，需协同优化计算与通信资源，确保高效资源分配与无缝服务交付。本文提出统一框架，全面综述在异构设备上部署FM驱动代理服务的前沿进展，重点聚焦模型与资源优化的深度融合，构建稳健基础设施。首先探讨推理阶段的底层优化策略，分析提升系统可扩展性的方法，如并行化技术与资源弹性伸缩。随后研究主流大模型及其推理加速技术，包括模型压缩与令牌缩减。同时剖析构建代理服务的关键组件，并展示典型智能应用场景。最后，展望实时代理服务中高服务质量（QoS）实现的潜在研究方向。
原文摘要 · Abstract (English)
Foundation model (FM) powered agent services are regarded as a promising solution to develop intelligent and personalized applications for advancing toward Artificial General Intelligence (AGI). To achieve high reliability and scalability in deploying these agent services, it is essential to collaboratively optimize computational and communication resources, thereby ensuring effective resource allocation and seamless service delivery. In pursuit of this vision, this paper proposes a unified framework aimed at providing a comprehensive survey on deploying FM-based agent services across heterogeneous devices, with the emphasis on the integration of model and resource optimization to establish a robust infrastructure for these services. Particularly, this paper begins with exploring various low-level optimization strategies during inference and studies approaches that enhance system scalability, such as parallelism techniques and resource scaling methods. The paper then discusses several prominent FMs and investigates research efforts focused on inference acceleration, including techniques such as model compression and token reduction. Moreover, the paper also investigates critical components for constructing agent services and highlights notable intelligent applications. Finally, the paper presents potential research directions for developing real-time agent services with high Quality of Service (QoS).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。