为大模型推理的两个阶段定制专用硬件，降低成本与功耗。
SPAD: Specialized Prefill and Decode Hardware for Disaggregated LLM Inference
- 针对预填充和解码阶段设计不同芯片，分别优化计算与内存性能。
- 预填充芯片成本降52%且性能提升8%，解码芯片功耗降28%仍保持97%性能。
- 可灵活调配芯片类型，适应不同模型和负载，长期成本优势明显。
大型语言模型（LLMs）的普及推动了推理需求的增长。其推理过程分为计算密集型的预填充阶段和内存密集型的解码阶段。现有数据中心GPU和TPU采用‘越多越好’的设计理念，导致预填充阶段内存带宽利用率低、解码阶段计算资源浪费，从而增加服务成本。本文提出SPAD（专用预填充与解码硬件），采用‘少即是多’策略，为两阶段分别设计专用芯片：预填充芯片配备更大阵列和低成本GDDR内存，解码芯片保持高内存带宽但降低计算能力。模拟结果显示，相比建模的H100，预填充芯片平均预填充性能提升8%、硬件成本降低52%；解码芯片实现97%解码性能，功耗降低28%。端到端仿真在生产数据集上显示，相较于基线集群，SPAD可降低19%-41%硬件成本和2%-17%功耗，同时保持相同性能。即使模型与工作负载变化，通过重新分配芯片类型仍能实现11%-43%的成本下降，验证了该设计的长期适用性。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have gained popularity in recent years, driving up the demand for inference. LLM inference is composed of two phases with distinct characteristics: a compute-bound prefill phase followed by a memory-bound decode phase. To efficiently serve LLMs, prior work proposes prefill-decode disaggregation to run each phase on separate hardware. However, existing hardware poorly matches the different requirements of each phase. Current datacenter GPUs and TPUs follow a more-is-better design philosophy that maximizes compute and memory resources, causing memory bandwidth underutilization in the prefill phase and compute underutilization in the decode phase. Such underutilization directly translates into increased serving costs. This paper proposes SPAD (Specialized Prefill and Decode hardware), adopting a less-is-more methodology to design specialized chips tailored to the distinct characteristics of prefill and decode phases. The proposed Prefill Chips have larger systolic arrays and use cost-effective GDDR memory, whereas the proposed Decode Chips retain high memory bandwidth but reduce compute capacity. Compared to modeled H100s, simulations show that the proposed Prefill Chips deliver 8% higher prefill performance on average at 52% lower hardware cost, while the proposed Decode Chips achieve 97% of the decode performance with 28% lower TDP. End-to-end simulations on production traces show that SPAD reduces hardware cost by 19%-41% and TDP by 2%-17% compared to modeled baseline clusters while offering the same performance. Even when models and workloads change, SPAD can reallocate either type of chip to run either phase and still achieve 11%-43% lower hardware costs, demonstrating the longevity of the SPAD design.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。