为中小企业部署本地大模型提供硬件软件协同方案
On-Device LLMs for SMEs: Challenges and Opportunities
- 从软硬件双角度分析企业本地部署大模型的适配路径
- 指出资源受限环境下算力与存储优化的关键挑战
- 适合关注边缘AI落地的中小企业技术决策者
本文系统性地回顾了在小型和中型企业（SMEs）环境中，将大型语言模型（LLMs）本地部署所需的基础设施要求，涵盖硬件与软件两个层面。从硬件视角出发，讨论了GPU、TPU等处理单元的使用，高效内存与存储解决方案，以及在计算资源有限的典型中小企业场景下，实现有效部署的策略。从软件角度看，探讨了框架兼容性、操作系统优化，以及针对资源受限环境设计的专用库的应用。该综述首先识别了中小企业在本地部署LLMs时面临的独特挑战，随后探索了硬件创新与软件适配所带来的机遇，为克服这些障碍提供了可能。这种结构化综述为社区提供了实用洞见，显著增强了中小企业在集成LLMs方面的技术韧性。
原文摘要 · Abstract (English)
This paper presents a systematic review of the infrastructure requirements for deploying Large Language Models (LLMs) on-device within the context of small and medium-sized enterprises (SMEs), focusing on both hardware and software perspectives. From the hardware viewpoint, we discuss the utilization of processing units like GPUs and TPUs, efficient memory and storage solutions, and strategies for effective deployment, addressing the challenges of limited computational resources typical in SME settings. From the software perspective, we explore framework compatibility, operating system optimization, and the use of specialized libraries tailored for resource-constrained environments. The review is structured to first identify the unique challenges faced by SMEs in deploying LLMs on-device, followed by an exploration of the opportunities that both hardware innovations and software adaptations offer to overcome these obstacles. Such a structured review provides practical insights, contributing significantly to the community by enhancing the technological resilience of SMEs in integrating LLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。