LIME让推荐系统在长序列和大候选集下仍保持高效，推理速度提升10倍。
LIME: Link-based user-item Interaction Modeling with decoupled xor attention for Efficient test time scaling
- 用低秩链接嵌入解耦用户与物品交互，预计算注意力权重。
- 线性注意力机制将序列复杂度从O(N²)降至O(N)，支持长序列处理。
- 适合需要高并发、低延迟的工业级推荐场景，如电商、内容平台。
扩展大规模推荐系统需突破三个关键瓶颈：处理更长的用户历史、扩大候选集规模、提升模型容量。尽管前景广阔，但传统Transformer的计算开销随用户序列长度呈平方增长，随候选数线性增长，导致在推理阶段扩展候选集或延长序列成本过高。本文提出新型架构LIME，通过两项核心创新解决该权衡问题。首先，低秩“链接嵌入”实现用户与候选项交互的解耦，可预计算注意力权重，使推理成本几乎与候选集大小无关；其次，引入线性注意力机制LIME-XOR，将用户序列复杂度从二次方O(N²)降至线性O(N)。在公开及工业数据集上的实验表明，LIME在性能接近当前最优Transformer的同时，在大候选集或长序列场景下实现10倍推理加速。在主流推荐平台测试中，LIME显著提升用户参与度，且对候选集大小和用户历史长度的推理开销近乎恒定，为高效且表达能力强的推荐系统树立新范式。
原文摘要 · Abstract (English)
Scaling large recommendation systems requires advancing three major frontiers: processing longer user histories, expanding candidate sets, and increasing model capacity. While promising, transformers' computational cost scales quadratically with the user sequence length and linearly with the number of candidates. This trade-off makes it prohibitively expensive to expand candidate sets or increase sequence length at inference, despite the significant performance improvements. We introduce \textbf{LIME}, a novel architecture that resolves this trade-off. Through two key innovations, LIME fundamentally reduces computational complexity. First, low-rank ``link embeddings" enable pre-computation of attention weights by decoupling user and candidate interactions, making the inference cost nearly independent of candidate set size. Second, a linear attention mechanism, \textbf{LIME-XOR}, reduces the complexity with respect to user sequence length from quadratic ($O(N^2)$) to linear ($O(N)$). Experiments on public and industrial datasets show LIME achieves near-parity with state-of-the-art transformers but with a 10$\times$ inference speedup on large candidate sets or long sequence lengths. When tested on a major recommendation platform, LIME improved user engagement while maintaining minimal inference costs with respect to candidate set size and user history length, establishing a new paradigm for efficient and expressive recommendation systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。