让大模型推荐生成更快:通过位置感知提升预测准确性
Position-Aware Drafting for Inference Acceleration in LLM-Based Generative List-Wise Recommendation
- 引入位置感知机制,区分令牌在物品内的位置和推测深度
- 实测最高提速3.1倍,平均提速约5%,推荐质量基本不变
- 模块轻量易集成,适合需要低延迟的推荐系统场景
基于大语言模型的生成式列表推荐发展迅速,但解码仍为串行过程,导致延迟高。为加速推理而不改变目标分布,推测解码(SD)使用小型草稿模型一次性提出多个下一令牌,由目标LLM验证并接受最长前缀,从而跳过多步。但在生成式推荐中,每个物品由多个语义-标识符令牌表示,常带分隔符,现有草稿模型通常对这些令牌一视同仁。这忽略了两个现实问题:(i) 令牌语义依赖其在物品内的槽位,(ii) 推测深度增加时不确定性上升。若不建模这些因素,SD的加速效果受限。本文提出PAD-Rec,一种位置感知草稿模块,通过两个互补信号增强草稿模型:物品位置嵌入显式编码每个令牌的物品内槽位,强化结构感知;步骤位置嵌入编码推测步骤,使模型能适应深度相关的不确定性,提升预测质量。为融合这些信号与基础特征,加入简单门控机制:一个可学习的物品槽位系数,以及一个上下文驱动的步骤门控。该模块可训练、易于集成标准草稿模型,且推理开销极小。在四个真实数据集上的大量实验表明,最高实现3.1倍的墙钟速度提升,相比强基线平均提速约5%,同时基本保持推荐质量。
原文摘要 · Abstract (English)
Large language model (LLM)-based generative list-wise recommendation has advanced rapidly, but decoding remains sequential and thus latency-prone. To accelerate inference without changing the target distribution, speculative decoding (SD) uses a small draft model to propose several next tokens at once and a target LLM to verify and accept the longest prefix, skipping multiple steps per round. In generative recommendation, however, each item is represented by multiple semantic-ID tokens, often with separators, and current drafts typically treat these tokens uniformly. This overlooks two practical facts: (i) a token's semantics depend on its within-item slot, and (ii) uncertainty tends to increase with speculation depth. Without modeling these effects, SD's speedups can be limited. We introduce PAD-Rec, Position-Aware Drafting for generative Recommendation, a lightweight module that augments the draft model with two complementary signals. Item position embeddings explicitly encode the within-item slot of each token, strengthening structural awareness. Step position embeddings encode the draft step, allowing the model to adapt to depth-dependent uncertainty and improve proposal quality. To harmonize these signals with base features, we add simple gates: a learnable coefficient for item slots and a context-driven gate for draft steps. The module is trainable, easy to integrate with standard draft models, and adds negligible inference overhead. Extensive experiments on four real-world datasets show up to 3.1x wall-clock speedup and about 5% average wall-clock speedup gain over strong SD baselines, while largely preserving recommendation quality.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。