梳理大模型时代软硬件协同设计的挑战与方案。
A Survey: Collaborative Hardware and Software Design in the Era of Large Language Models
- 分析大模型对算力、能耗的独特需求
- 总结软硬件协同优化的关键技术路径
- 适合关注AI系统架构的工程师与研究者
大语言模型（LLMs）的快速发展显著推动了人工智能的进步，在自然语言处理领域展现出强大能力，并逐步拓展至多模态应用。这些模型在科研与产业界得到广泛应用，但其研发与部署面临巨大挑战：需要大量计算资源、高能耗，以及复杂的软件优化。与传统深度学习系统不同，大语言模型需针对训练与推理特点进行系统级效率优化。本文综述了专为大语言模型特性与约束设计的软硬件协同设计方法，分析其对硬件与算法研究的影响，探讨算法优化、硬件设计与系统创新。旨在全面理解大模型驱动的计算系统中的权衡与考量，指导下一代AI系统的演进。最后，总结现有工作并展望未来方向，推动面向生产级应用的大模型协同设计方法发展。
原文摘要 · Abstract (English)
The rapid development of large language models (LLMs) has significantly transformed the field of artificial intelligence, demonstrating remarkable capabilities in natural language processing and moving towards multi-modal functionality. These models are increasingly integrated into diverse applications, impacting both research and industry. However, their development and deployment present substantial challenges, including the need for extensive computational resources, high energy consumption, and complex software optimizations. Unlike traditional deep learning systems, LLMs require unique optimization strategies for training and inference, focusing on system-level efficiency. This paper surveys hardware and software co-design approaches specifically tailored to address the unique characteristics and constraints of large language models. This survey analyzes the challenges and impacts of LLMs on hardware and algorithm research, exploring algorithm optimization, hardware design, and system-level innovations. It aims to provide a comprehensive understanding of the trade-offs and considerations in LLM-centric computing systems, guiding future advancements in AI. Finally, we summarize the existing efforts in this space and outline future directions toward realizing production-grade co-design methodologies for the next generation of large language models and AI systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。