提出高效验证的LLM注意力机制，显著降低远程推理通信开销。
Communication-Efficient Verifiable Attention for LLM Inference
- 将注意力计算全量卸载至GPU，TEE仅负责验证，减少通信与计算开销。
- 预填充阶段采用双级流水线，重叠数据传输与计算，加速2.60-3.38倍。
- 解码时动态分片处理缓存溢出，避免重复传输，提速3.86-5.42倍。
远程大语言模型服务的计算完整性存在疑虑。传统基于可信执行环境（TEE）的深度神经网络分片方法（TSDP）虽可验证线性组件，但直接用于Transformer架构的LLM时，会带来显著的TEE计算与通信开销。本文提出通信高效的TEE-GPU注意力机制（VeriAttn），将注意力的线性与非线性部分全部卸载至GPU，由TEE完成验证。针对预填充阶段，设计两级流水线以重叠数据移动、TEE预/后处理与GPU计算；针对解码阶段，当键值缓存超出GPU内存时，将注意力计算跨分片至TEE与GPU，减少重复键值传输。在Intel TDX平台上评估显示，对于6k token提示和10k token输出，VeriAttn相比TSDP在预填充和解码阶段分别实现2.60-3.38×和3.86-5.42×的加速。
原文摘要 · Abstract (English)
Computation integrity of remote large language model (LLM) serving can be questionable. For conventional deep neural networks (DNNs), the existing TEE-shielded DNN partitioning (TSDP) approach uses Trusted Execution Environment (TEE) to compute non-linear components and verify the integrity of linear components offloaded to an untrusted GPU. However, directly applying TSDP to Transformer-based LLMs incurs significant TEE computation and TEE-GPU communication overhead. This paper presents Communication-efficient TEE-GPU Attention (\textsc{VeriAttn}) for accelerating verifiable LLM inference. \textsc{VeriAttn} offloads both linear and non-linear computations of attention to the GPU, while TEE performs verification. Moreover, for prefill, \textsc{VeriAttn} uses a two-level pipeline to overlap data movement, TEE pre-/post-processing, and GPU computation. For decoding, when the key-value cache exceeds available GPU memory, \textsc{VeriAttn} partitions attention across TEE and GPU to reduce repeated key-value transfers. Evaluation on an Intel TDX platform shows that \textsc{VeriAttn} achieves 2.60-3.38$\times$ and 3.86-5.42$\times$ acceleration over TSDP for 6k-token prompts and 10k-token outputs during prefill and decoding, respectively.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。