arXiv:2410.23300cs.IRcs.LG2024-10被引 10

通过矩阵稳定秩分析,提出高效正则化方法提升推荐系统训练速度与性能。

Understanding and Scaling Collaborative Filtering Optimization from the Perspective of Matrix Rank

  • 从矩阵稳定秩视角揭示协同过滤优化机制,发现损失函数与奇异值分布的内在联系。
  • 早期训练阶段正则化稳定秩,使训练速度提升最高达66%,性能超越小负样本率模型21%。
  • 该方法可替代负采样,适合追求高效率和高性能推荐系统的开发者使用。

协同过滤(CF)方法在真实推荐系统中占据主导地位,因其能学习高质量、稀疏的用户-物品嵌入表,有效捕捉用户偏好。这些嵌入表随用户和物品数量线性增长,训练目标是增强交互对的嵌入相似性,同时降低未交互对的相似性。然而,为实现非交互对的充分分散,需依赖昂贵的正则化(如负采样),影响运行效率与可扩展性。现有研究多通过简化模型或采样数据来缓解问题,牺牲性能以换取速度。本文超越模型层面修改,从不同学习策略下嵌入表的性质出发进行研究。理论分析表明,嵌入表的奇异值与不同CF损失函数密切相关。在真实数据集上实证验证了高稳定秩(矩阵秩的连续版本,反映奇异值分布)的实用性。基于此,提出一种高效的预热正则化策略,对用户和物品嵌入的稳定秩进行约束。结果表明,早期训练阶段引入稳定秩正则化可生成更高质量嵌入,训练速度最高提升66%;同时可作为负采样的替代,使性能相比小负样本率损失函数提升最高达21%。整体分析统一了现有CF方法的优化本质——稳定秩优化,启发了一种灵活的正则化方法。

原文摘要 · Abstract (English)

Collaborative Filtering (CF) methods dominate real-world recommender systems given their ability to learn high-quality, sparse ID-embedding tables that effectively capture user preferences. These tables scale linearly with the number of users and items, and are trained to ensure high similarity between embeddings of interacted user-item pairs, while maintaining low similarity for non-interacted pairs. Despite their high performance, encouraging dispersion for non-interacted pairs necessitates expensive regularization (e.g., negative sampling), hurting runtime and scalability. Existing research tends to address these challenges by simplifying the learning process, either by reducing model complexity or sampling data, trading performance for runtime. In this work, we move beyond model-level modifications and study the properties of the embedding tables under different learning strategies. Through theoretical analysis, we find that the singular values of the embedding tables are intrinsically linked to different CF loss functions. These findings are empirically validated on real-world datasets, demonstrating the practical benefits of higher stable rank, a continuous version of matrix rank which encodes the distribution of singular values. Based on these insights, we propose an efficient warm-start strategy that regularizes the stable rank of the user and item embeddings. We show that stable rank regularization during early training phases can promote higher-quality embeddings, resulting in training speed improvements of up to 66%. Additionally, stable rank regularization can act as a proxy for negative sampling, allowing for performance gains of up to 21% over loss functions with small negative sampling ratios. Overall, our analysis unifies current CF methods under a new perspective, their optimization of stable rank, motivating a flexible regularization method.

协同过滤矩阵优化推荐系统稳定秩

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。