解决推荐系统持续学习中的知识刷新难题,提升长期效果。
Knowledge-Geometry Decoupling: Refreshable Pretrained Transfer for Streaming Recommendation
- 用多目标行为预测保留相关序列,避免错误关联。
- 分离知识与任务几何结构,支持模型持续更新而不干扰下游任务。
- 在真实电商场景中提升1.75%用户营收,适合工业级推荐系统使用。
工业推荐系统广泛采用预训练-迁移范式,但行为分布漂移带来两个问题:应从行为序列中学习什么,以及如何在持续刷新预训练模型时有效迁移知识。为此,我们提出知识-几何解耦(KGD)。针对学习内容,传统下一标记预测将邻接视为依赖,可能编码无关会话间的虚假转移;我们引入行为多标记预测(BMTP),仅以协同或语义相关未来项目为监督信号,获得更清洁、可迁移的行为知识。针对迁移方式,预训练知识与任务特定几何结构对共享参数有冲突优化需求;KGD将其分配至独立参数集:可刷新编码器负责行为知识,任务学习器通过只读交叉注意力读取上下文编码状态,并通过正交于预训练嵌入的锚定校准残差(ACR)写入任务几何结构。解耦机制使知识可持续刷新,且不产生任务梯度干扰或破坏下游适应。KGD在八个公开基准上比强基线提升4%-12%,并在90天线上流数据中持续保持优势,而基线无增益。该模型已在虾皮(Shopee)全面部署,在主页搜索的实时A/B测试中,用户每单GMV提升1.75%,广告收入提升1.53%,验证其高实用价值。核心代码已开源:https://github.com/FuCongResearchSquad/KGD4REC。
原文摘要 · Abstract (English)
Industrial recommenders increasingly adopt the pretrain-then-transfer paradigm, yet behavioral distribution drift raises two questions: what to learn from behavior sequences, and how to transfer the learned knowledge while the pretrained model is continually refreshed. To resolve them, we propose Knowledge-Geometry Decoupling (KGD). For what to learn, conventional next-token prediction treats adjacency as dependency and may encode spurious transitions across unrelated sessions. We introduce Behavioral Multi-Token Prediction (BMTP) to retain only collaboratively or semantically related future items as supervision, yielding cleaner and more transferable behavioral knowledge. For how to transfer, pretrained knowledge and task-specific geometry impose conflicting optimization demands on shared parameters. To handle it, KGD assigns them to separate parameter sets: a refreshable encoder owns behavioral knowledge, while a task learner reads contextualized encoder states through read-only cross-attention and writes task-specific geometry through Anchored Calibration Residual (ACR) orthogonal to the pretrained embedding. The decoupled ownership enables continual knowledge refresh without task-gradient interference or invalidating downstream adaptation. KGD improves over strong pretrain-transfer baselines by 4-12% on eight public benchmarks and sustains its advantage over a 90-day production stream where baselines show no gains. KGD has been fully deployed in Shopee. In a live A/B test on Shopee Homepage Search, it increases GMV per user by 1.75% and advertising revenue by 1.53%, demonstrating its high practical value. We provide the core implementation of KGD at https://github.com/FuCongResearchSquad/KGD4REC.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。