用分组门控Transformer实现多维度用户建模,支持隐私保护的联邦学习。
Multifaceted User Modeling in Recommendation: A Federated Foundation Models Approach
- 设计分组门控Transformer层,逐层挖掘用户兴趣的多维度特征。
- 在多个推荐数据集上优于基线方法,显著提升用户建模精度。
- 支持联邦学习,可在保护隐私前提下融合多方私有数据。
多维度用户建模旨在从用户数据中发现细粒度模式并学习表征,揭示其多样化的兴趣与特征,如身份属性、偏好和人格特质。基于基础模型的推荐研究强调了Transformer架构在捕捉复杂非线性用户-物品交互关系方面的卓越能力。本文通过增强多维度用户建模能力,推进基于基础模型的推荐系统。我们提出一种专为推荐任务设计的新型Transformer层,利用自注意力机制捕获用户-物品交互序列模式。具体地,设计分组门控网络以识别用户群体,实现在不同层次上的分层发现,从而通过多层Transformer捕捉用户兴趣的多面性。此外,为拓展数据范围并进一步增强多维度用户建模能力,我们将框架扩展至联邦学习设置,支持使用私有数据集的同时保障隐私。在基准数据集上的实验验证了所提方法的优越性能。代码已公开。
原文摘要 · Abstract (English)
Multifaceted user modeling aims to uncover fine-grained patterns and learn representations from user data, revealing their diverse interests and characteristics, such as profile, preference, and personality. Recent studies on foundation model-based recommendation have emphasized the Transformer architecture's remarkable ability to capture complex, non-linear user-item interaction relationships. This paper aims to advance foundation model-based recommendersystems by introducing enhancements to multifaceted user modeling capabilities. We propose a novel Transformer layer designed specifically for recommendation, using the self-attention mechanism to capture sequential user-item interaction patterns. Specifically, we design a group gating network to identify user groups, enabling hierarchical discovery across different layers, thereby capturing the multifaceted nature of user interests through multiple Transformer layers. Furthermore, to broaden the data scope and further enhance multifaceted user modeling, we extend the framework to a federated setting, enabling the use of private datasets while ensuring privacy. Experimental validations on benchmark datasets demonstrate the superior performance of our proposed method. Code is available.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。