为大模型推理设计专用芯片，显著提升内存带宽效率。
RPU -- A Reasoning Processing Unit
- 采用容量优化高带宽内存与分立式微架构，解耦计算、通信与存储
- 在同功耗下比H100快45.3倍，吞吐量提升18.6倍
- 适合长序列生成、低算力密度的推理场景
大语言模型推理性能正受内存墙制约。尽管GPU持续提升计算吞吐量，但在内存带宽受限的任务中难以实现可扩展性。这一问题在新兴的推理类应用中尤为突出：长输出序列、低算术强度和严格的延迟要求导致对内存带宽需求激增，系统利用率下降，每推理能耗上升。为此，本文提出推理处理单元（RPU），一种基于芯片组的架构设计，以应对现代内存墙挑战。RPU引入三项创新：(1) 容量优化型高带宽内存（HBM-CO），以牺牲容量换取更低功耗和成本；(2) 带宽优先的可扩展芯片组架构，按需分配功耗与面积；(3) 解耦的微架构，将内存、计算和通信管道分离，维持高带宽利用率。仿真结果显示，在同功耗（ISO-TDP）条件下，RPU对Llama3-405B的推理延迟降低45.3倍，吞吐量提升18.6倍。
原文摘要 · Abstract (English)
Large language model (LLM) inference performance is increasingly bottlenecked by the memory wall. While GPUs continue to scale raw compute throughput, they struggle to deliver scalable performance for memory bandwidth bound workloads. This challenge is amplified by emerging reasoning LLM applications, where long output sequences, low arithmetic intensity, and tight latency constraints demand significantly higher memory bandwidth. As a result, system utilization drops and energy per inference rises, highlighting the need for an optimized system architecture for scalable memory bandwidth. To address these challenges we present the Reasoning Processing Unit (RPU), a chiplet-based architecture designed to address the challenges of the modern memory wall. RPU introduces: (1) A Capacity-Optimized High-Bandwidth Memory (HBM-CO) that trades capacity for lower energy and cost; (2) a scalable chiplet architecture featuring a bandwidth-first power and area provisioning design; and (3) a decoupled microarchitecture that separates memory, compute, and communication pipelines to sustain high bandwidth utilization. Simulation results show that RPU performs up to 45.3x lower latency and 18.6x higher throughput over an H100 system at ISO-TDP on Llama3-405B.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。