从演示系统到可量产的模型软件,破解部署难题
From Cool Demos to Production-Ready FMware: Core Challenges and a Technology Roadmap
- 梳理工业界与学术界实践,提炼生产级模型软件核心挑战
- 覆盖选型、对齐、提示工程、编排等全流程关键问题
- 适合企业AI落地、平台开发及标准制定者参考
基础模型(FM)的快速发展催生了以模型为核心组件的FMware系统。尽管构建演示级系统相对简单,但实现生产就绪仍面临可靠性、高成本、可扩展性及隐私合规等多重挑战。本文通过半结构化主题分析,结合我们开发FMArts——一个面向模型生命周期的工程平台并集成至华为云的经验,以及参与OPEA开源平台、组织AIware会议与培训、共同领导ISO SPDX SBOM AI工作组的实践,系统识别出在多源数据场景下生产化FMware的关键障碍,包括模型选择、数据与模型对齐、提示工程、智能体编排、系统测试与部署等问题,同时涵盖内存管理、可观测性与反馈集成等跨领域议题。文中讨论了应对这些挑战所需的技术与策略,并为从演示系统迈向可扩展、生产就绪的FMware提供指引。研究强调持续研究与跨行业协作对推动生产级FMware发展的重要性。
原文摘要 · Abstract (English)
The rapid expansion of foundation models (FMs), such as large language models (LLMs), has given rise to FMware, software systems that integrate FM(s) as core components. While building demonstration-level FMware is relatively straightforward, transitioning to production-ready systems presents numerous challenges, including reliability, high implementation costs, scalability, and compliance with privacy regulations. Our paper conducts a semi-structured thematic synthesis to identify key challenges in productionizing FMware across diverse data sources, including our industry experience developing FMArts, a FMware lifecycle engineering platform, and its integration into Huawei Cloud; grey literature; academic publications; hands-on involvement in the Open Platform for Enterprise AI (OPEA); organizing the AIware conference and bootcamp; and co-leading the ISO SPDX SBOM working group on AI and datasets. We identify critical issues in FM(s) selection, data and model alignment, prompt engineering, agent orchestration, system testing, and deployment, alongside cross-cutting concerns such as memory management, observability, and feedback integration. We discuss necessary technologies and strategies to address these challenges and offer guidance to enable the transition from demonstration systems to scalable, production-ready FMware solutions. Our findings underscore the importance of continued research and multi-industry collaboration to advance the development of production-ready FMware.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。