arXiv:2508.10401cs.IR2025-08被引 2

用代理模型+强化学习,智能选用户参与推荐系统训练。

Proxy Model-Guided Reinforcement Learning for Client Selection in Federated Recommendation

  • 用轻量代理模型估算用户贡献,省去昂贵的本地训练。
  • 结合准确率与嵌入新鲜度设计奖励函数,提升推荐效果。
  • 适合大规模、数据稀疏的推荐场景,提升模型更新质量。

联邦推荐系统在保护用户隐私的同时实现个性化推荐,但现有方法多采用随机选用户策略,忽视用户数据的统计异质性,导致模型性能不佳。尽管已有客户端选择策略,但大多针对通用任务,难以应对推荐场景中客户数量大、贡献评估成本高、长尾物品分布导致更新稀疏等问题。为此,我们提出ProxyRL-FRS,一种面向联邦推荐的代理模型引导强化学习框架。首先,在每个客户端部署双分支模型ProxyNCF,通过额外的轻量代理分支实现快速贡献估计,避免每轮需进行完整本地训练的开销。其次,设计一种考虑延迟的强化学习代理(SA),基于代理模型估算的贡献选择客户端,并以兼顾推荐精度与嵌入新鲜度的奖励函数进行优化,从而提升物品嵌入的更新覆盖率。在多个公开推荐数据集上的实验验证了该方法的有效性。

原文摘要 · Abstract (English)

Federated recommender systems have emerged as a promising privacy-preserving paradigm, enabling personalized recommendation services without exposing users' raw data. By keeping data local and relying on a central server to coordinate training across distributed clients, FedRSs protect user privacy while collaboratively learning global models. However, most existing FedRS frameworks adopt fully random client selection strategy in each training round, overlooking the statistical heterogeneity of user data arising from diverse preferences and behavior patterns, thereby resulting in suboptimal model performance. While some client selection strategies have been proposed in the broader federated learning literature, these methods are typically designed for generic tasks and fail to address the unique challenges of recommendation scenarios, such as expensive contribution evaluation due to the large number of clients, and sparse updates resulting from long-tail item distributions. To bridge this gap, we propose ProxyRL-FRS, a proxy model-guided reinforcement learning framework tailored for client selection in federated recommendation. Specifically, we first introduce ProxyNCF, a dual-branch model deployed on each client, which augments standard Neural Collaborative Filtering with an additional proxy model branch that provides lightweight contribution estimation, thus eliminating the need for expensive per-round local training traditionally required to evaluate a client's contribution. Furthermore, we design a staleness-aware SA reinforcement learning agent that selects clients based on the proxy-estimated contribution, and is guided by a reward function balancing recommendation accuracy and embedding staleness, thereby enriching the update coverage of item embeddings. Experiments conducted on public recommendation datasets demonstrate the effectiveness of ProxyRL-FRS.

联邦学习推荐系统强化学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。