LLM推理硬件面临内存与通信瓶颈，提出四类架构优化方向。
Challenges and Research Directions for Large Language Model Inference Hardware
- 针对推理阶段特性，聚焦内存带宽与互连延迟问题。
- 提出高带宽闪存可实现10倍内存容量与类HBM带宽。
- 适合数据中心与移动设备的低延迟硬件设计参考。
大语言模型（LLM）推理极具挑战性。底层Transformer模型的自回归解码阶段使推理在本质上区别于训练。受近期AI趋势加剧影响，主要瓶颈已从算力转向内存和互连。为应对这些挑战，我们提出四大架构研究机遇：高带宽闪存可实现10倍内存容量并具备类HBM的带宽；近内存计算与3D堆叠内存-逻辑集成以提升内存带宽；以及低延迟互连以加速通信。尽管我们的重点是数据中心AI，但也评估了这些方案在移动设备上的适用性。
原文摘要 · Abstract (English)
Large Language Model (LLM) inference is hard. The autoregressive Decode phase of the underlying Transformer model makes LLM inference fundamentally different from training. Exacerbated by recent AI trends, the primary challenges are memory and interconnect rather than compute. To address these challenges, we highlight four architecture research opportunities: High Bandwidth Flash for 10X memory capacity with HBM-like bandwidth; Processing-Near-Memory and 3D memory-logic stacking for high memory bandwidth; and low-latency interconnect to speedup communication. While our focus is datacenter AI, we also review their applicability for mobile devices.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。