CHIME通过整合DIMM-PIM实现高效长上下文注意力计算，突破内存瓶颈。
CHIME: A Case for Efficient Long-Context Attention-FC Disaggregated Inference with DIMM-PIM
- 将注意力计算卸载至高带宽的DIMM-PIM，保留全连接层在GPU上运行。
- 采用无气泡流水线与混合粒度重排，提升分布式内存芯片协同效率。
- 适合追求长序列推理性能的系统设计者，尤其关注内存-计算协同优化者。
注意力-全连接解耦（AFD）大模型推理系统将内存密集型注意力操作卸载至内存丰富的加速器（如CPU、HBM-PIM），而将计算密集型全连接操作保留在GPU上。本文首先构建了分解式屋顶模型（DRM）来刻画AFD性能，揭示系统吞吐受限于加速器的瓶颈因素：内存带宽或容量。观察发现，先前AFD系统常忽视这些约束，导致资源利用率不足或吞吐受限。为此，我们提出CHIME，首个集成DIMM-PIM的AFD系统，该新型加速器具备可扩展的容量与带宽。为应对DIMM-PIM中分布式DRAM芯片的同步挑战，CHIME采用无气泡流水线与混合粒度重排以高效完成注意力计算。同时，通过秩集粒度的通信-计算重叠和对齐预测调度，最大化跨设备资源利用率。评估显示，CHIME相较最先进HBM-PIM方案最高提速5.15倍。
原文摘要 · Abstract (English)
Attention-FC Disaggregated (AFD) LLM inference systems offload memory-bound Attention operations to memory-rich accelerators (e.g., CPUs, HBM-PIM) while retaining compute-bound Fully-Connected (FC) operations on GPUs. In this paper, we first design a Disaggregated Roofline Model (DRM) to characterize AFD performance, revealing that system throughput is constrained by the accelerator's limiting factor: either memory bandwidth or capacity. We observe that prior AFD systems often overlook these constraints and fail to balance them, leading to resource underutilization or constrained throughput. Therefore, we propose CHIME, the first AFD system integrating DIMM-PIM, which is a case of the new accelerator that strikes the balance with scalable capacity and bandwidth. To address the synchronization challenges inherent to the distributed cooperating DRAM chips in DIMM-PIM, CHIME employs bubble-free pipelining and hybrid-grained re-layout for efficient attention computation. Furthermore, it maximizes cross-device resource utilization via rankset-granular communication-computation overlapping and alignment-predicting scheduling. Evaluations show CHIME achieves up to 5.15$\times$ speedup over state-of-the-art HBM-PIM solutions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。