arXiv:2507.00715cs.IR2025-07KDD被引 16

通过注册令牌压缩序列,实现大模型推荐的高效推理

EARN: Efficient Inference Acceleration for LLM-based Generative Recommendation by Register Tokens

  • 用早期层信息生成边界注册令牌,替代完整序列输入
  • 实测最高提速3.79倍,缓存减少80.8%且精度更高
  • 适合工业级大模型推荐系统部署,兼顾速度与效果

基于大语言模型的生成式推荐(LLMRec)虽表现优异,但因键值缓存(KV Cache)带来的高计算开销和内存压力导致推理延迟高。现有缓存压缩方法存在局限:压缩效果有限,且提示词压缩易丢失关键交互历史。通过对LLMRec中注意力模式的系统分析,我们发现两个关键现象:1)层间注意力稀疏性反转,即早期层信息密集而后期层冗余度高;2)双注意力聚集,注意力集中在序列头尾令牌上。受此启发,我们提出EARN框架:利用早期层将信息压缩为置于序列边界的注册令牌,并在后续层仅处理这些令牌。在三个数据集、两种LLMRec方法和两种LLM架构上的实验表明,EARN可实现最高3.79倍加速,减少80.8%的KV Cache,同时精度优于通用微调方法。本工作弥合了LLMRec中效率与效果的鸿沟,为工业场景提供实用部署优势。

原文摘要 · Abstract (English)

Large Language Model-based generative recommendation (LLMRec) has achieved notable success, but it suffers from high inference latency due to massive computational overhead and memory pressure of KV Cache. Existing KV Cache reduction methods face critical limitations: cache compression offers marginal acceleration given recommendation tasks' short decoding steps, while prompt compression risks discarding vital interaction history. Through systematic analysis of attention patterns in LLMRec, we uncover two pivotal insights: 1) layer-wise attention sparsity inversion where early layers retain dense informative patterns while later layers exhibit high redundancy, and 2) dual attention sinks phenomenon where attention scores concentrate on both head and tail tokens of input sequences. Motivated by these insights, we propose EARN, an efficient inference framework that leverages the early layers to compress information into register tokens placed at the input sequence boundaries, then focuses solely on these tokens in the subsequent layers. Extensive experiments on three datasets, two LLMRec methods and two LLM architectures demonstrate EARN's superiority, achieving up to 3.79x speedup and 80.8% KV Cache reduction with better accuracy than the general finetuning approach. Our work bridges the efficiency-effectiveness gap in LLMRec, offering practical deployment advantages for industrial scenarios.

大模型推荐推理加速缓存优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。