解决推荐系统中异构特征导致的模型扩展瓶颈。
TransRetrieval: Scaling Up Transformer-Based Retrieval for Industrial Recommendation
- 用加权平均聚合恢复Transformer所需的均匀令牌假设。
- 压缩目标令牌，降低85%计算量仍保持注意力表达能力。
- 通过领域嵌入统一多域数据，适合大规模工业部署。
将缩放定律应用于推荐检索受限于特征异质性：简单堆叠Transformer层会导致收益递减，因异构字段引发严重令牌范数差异。我们提出TransRetrieval，一种可随计算预算与跨域数据规模扩展的Transformer检索框架。核心是（1）加权平均聚合，恢复Transformer依赖的均一令牌假设；基于此，引入（2）目标令牌压缩技术，使每候选项计算量减少85%的同时保持交叉注意力表达力；（3）位置风格的领域嵌入，在几乎无额外开销下统一多个领域，使跨域数据成为扩展资产。在含400亿交互行为的工业数据集及公开的KuaiRand基准上，计算量从0.1增至2 MFLOPs/目标时，Recall@2000提升19.3/+22.2个百分点，验证了稳健的对数线性缩放。在线A/B测试中，该模型在与生产基线相同端到端延迟约束下，提升平台收入2.53%。
原文摘要 · Abstract (English)
Applying scaling laws to recommendation retrieval is hindered by feature heterogeneity: naively stacking Transformer layers yields diminishing returns because heterogeneous fields produce severe token-norm divergence. We present TransRetrieval, a Transformer-based retrieval framework that scales with both computational budget and cross-domain data. The key enabler is (1) weighted average aggregation, which restores the homogeneous-token assumption Transformers rely on. Building on this, we introduce (2) target token compression that cuts per-candidate FLOPs by 85% while preserving cross-attention expressiveness, and (3) position-style domain embeddings that unify multiple domains at negligible additional cost, turning cross-domain data into a scaling asset. On a 40-billion-interaction industrial dataset and the public KuaiRand benchmark, scaling compute from 0.1 to 2 MFLOPs per target yields +19.3/+22.2 pt Recall@2000, confirming robust log-linear scaling. In online A/B tests, TransRetrieval lifts platform revenue by 2.53% under the same end-to-end latency constraint as the production baseline.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。