arXiv:2606.21911cs.IRcs.LG2026-06KDD

模型越大越偏爱热门商品,新方法有效缓解这一问题。

The Pitfall of Scaling Up: Uncovering and Mitigating Popularity Bias Amplification in Scaling Transformer-based Recommenders

  • 通过约束注意力与前馈层,抑制模型深度带来的谱崩溃
  • 在0.05M到0.34B参数规模下显著提升长尾公平性
  • 适合追求公平推荐与可持续扩增的工业级推荐系统

我们发现,扩大基于Transformer的序列推荐模型虽能提升精度,却同时加剧了流行度偏差。该偏差导致系统过度推荐热门商品,牺牲小众商品,损害公平性并强化马太效应与信息茧房,成为可持续扩展的根本障碍。通过理论与实证分析,我们揭示根源:随着模型深度增加,注意力聚合与前馈投影协同引发严重的谱崩溃,直接导致流行度偏差放大。为此,我们提出SPRINT(Scalable Popularity Regularization IN Transformers),通过限制注意力矩阵的最大列和及前馈参数的谱范数,缓解谱崩溃。大量实验表明,SPRINT在0.05M至0.34B参数范围内显著提升准确率与长尾公平性,实现更优的可扩展性。代码已开源:https://github.com/Tiny-Snow/GenRec。

原文摘要 · Abstract (English)

We identify a critical pitfall in scaling transformer-based sequential recommenders: while increasing model size improves recommendation accuracy, it simultaneously amplifies popularity bias. This bias drives systems to over-recommend popular items at the expense of niche ones, which not only undermines fairness but also degrades the broader ecosystem by reinforcing the Matthew effect and filter bubbles. Consequently, this bias amplification emerges as a fundamental obstacle to sustainable model scaling. Through comprehensive theoretical and empirical analyses, we uncover the root cause of this amplification. Our findings reveal that as model depth increases, the two core components of the transformer architecture, i.e., attention aggregation and feed-forward projections, synergistically induce severe spectral collapse in model predictions, which directly translates to the amplification of popularity bias. To address this challenge, we propose SPRINT (Scalable Popularity Regularization IN Transformers), which mitigates spectral collapse during scaling by constraining (i) the maximum column-sums of the attention score matrices and (ii) the spectral norms of the feed-forward parameters. Extensive experiments demonstrate that SPRINT significantly improves both accuracy and long-tail fairness. Crucially, it yields more favorable scaling behaviors when expanding model sizes from 0.05M to 0.34B parameters. The code is available at https://github.com/Tiny-Snow/GenRec.

推荐系统流行度偏差Transformer可扩展性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。