用异构系统加速大模型长上下文推理中的内存处理，提升2.2倍速度并降低4.7倍能耗。
Understand and Accelerate Memory Processing Pipeline for Large Language Model Inference
- 将内存处理统一为四步流程，识别出22%-97%的性能开销
- 在GPU+FPGA上实现稀疏与内存密集操作卸载，速度提升2.2倍，能效降低4.7倍
- 适合追求高效推理的部署场景，尤其对长文本生成有显著收益
现代大语言模型日益依赖高效的长上下文处理与生成机制，包括稀疏注意力、检索增强生成（RAG）和压缩上下文记忆，以支持复杂推理。我们发现这些优化可统一为四步内存处理流水线：准备记忆、计算相关性、检索、应用到推理。通过系统性分析，我们识别出大模型推理中存在22%-97%的内存处理开销，且其计算特征具有强异质性。基于此，我们提出异构系统更适合加速内存处理，从而提升端到端推理效率。我们在GPU-FPGA系统上验证该思路：将稀疏、不规则且内存受限的操作卸载至FPGA，保留计算密集型任务于GPU。在AMD MI210 GPU与Alveo U55C FPGA上评估，系统速度最高达基线GPU的2.2倍，能效降低最高4.7倍；类似结果亦见于NVIDIA A100。这些成果确立了异构系统在高效大模型内存处理中的可行性，并为未来异构硬件设计提供指导。
原文摘要 · Abstract (English)
Modern large language models (LLMs) increasingly depends on efficient long-context processing and generation mechanisms, including sparse attention, retrieval-augmented generation (RAG), and compressed contextual memory, to support complex reasoning. We show that these optimizations can be unified into a four-step memory processing pipeline: Prepare Memory, Compute Relevancy, Retrieval, and Apply to Inference. Through systematic profiling, we identify a 22%-97% memory processing overhead in LLM inference and strong heterogeneity in its computational characteristics. Motivated by this insight, we argue that \textbf{heterogeneous systems} are well-suited to accelerate memory processing and thus end-to-end inference. We demonstrate this approach on a GPU-FPGA system by offloading sparse, irregular, and memory-bounded operations to FPGAs while retaining compute-intensive operations on GPUs. Evaluated on an AMD MI210 GPU and an Alveo U55C FPGA, our system is up to $2.2\times$ faster and achieves up to $4.7\times$ less energy across multiple LLM inference optimizations than the GPU baseline (similar results hold on NVIDIA A100). These results establish heterogeneous systems as a practical direction for efficient LLM memory processing and inform future heterogeneous hardware design.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。