用扩散模型增强双塔模型跨塔交互,提升大规模匹配精度
Unleashing the Potential of Two-Tower Models: Diffusion-Based Cross-Interaction for Large-Scale Matching
- 引入扩散模块重构用户下一意图,实现跨塔信息交互
- 在三个数据集上超越当前最优双塔模型,重建效果优于其他生成模型
- 适合大规模推荐系统优化,尤其关注用户意图建模的场景
双塔模型广泛应用于内容推荐、广告系统和搜索引擎等大规模匹配场景,通过分离用户与物品表征高效筛选候选集。然而,网络解耦也导致用户与物品表征间潜在交互被忽略。现有最先进方法包括添加浅层全连接层(COLD),但性能受限且仅适用于排序阶段;另一方法(DAT)通过将对方塔特征作为输入捕捉历史正向交互,但因缺乏对下一用户意图的引导,提升有限。为此,本文提出一种“交叉交互解耦架构”,在用户塔中引入扩散模块以重建下一正向意图表征,并采用混合注意力机制实现全面跨塔交互。在下一正向意图生成过程中,进一步显式提取用户行为序列中的时间漂移信息以提高重建准确性。在两个真实世界数据集和一个工业数据集上的实验表明,该方法显著优于当前最优双塔模型,其扩散方法在重建物品表征方面也优于其他生成模型。
原文摘要 · Abstract (English)
Two-tower models are widely adopted in the industrial-scale matching stage across a broad range of application domains, such as content recommendations, advertisement systems, and search engines. This model efficiently handles large-scale candidate item screening by separating user and item representations. However, the decoupling network also leads to a neglect of potential information interaction between the user and item representations. Current state-of-the-art (SOTA) approaches include adding a shallow fully connected layer(i.e., COLD), which is limited by performance and can only be used in the ranking stage. For performance considerations, another approach attempts to capture historical positive interaction information from the other tower by regarding them as the input features(i.e., DAT). Later research showed that the gains achieved by this method are still limited because of lacking the guidance on the next user intent. To address the aforementioned challenges, we propose a "cross-interaction decoupling architecture" within our matching paradigm. This user-tower architecture leverages a diffusion module to reconstruct the next positive intention representation and employs a mixed-attention module to facilitate comprehensive cross-interaction. During the next positive intention generation, we further enhance the accuracy of its reconstruction by explicitly extracting the temporal drift within user behavior sequences. Experiments on two real-world datasets and one industrial dataset demonstrate that our method outperforms the SOTA two-tower models significantly, and our diffusion approach outperforms other generative models in reconstructing item representations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。