arXiv:2605.24986cs.IRcs.LG2026-05

解决点击率预测中特征生成的难易不均问题,让模型更关注难学但重要的特征。

Self-Balancing Gradient Allocation for Heterogeneity-Aware Feature Generation in Click-Through Rate Prediction

论文配图:Self-Balancing Gradient Allocation for Heterogeneity-Aware Feature Generation in Click-Through Rate Prediction
图 1 · 摘自论文原文
  • 为每个特征字段引入可学习难度参数,动态调整训练梯度分配。
  • 在5个基准数据集和线上测试中显著提升点击率预测效果,尤其对冷启动用户改善明显。
  • 适合做推荐系统特征工程、点击率预估等需要处理稀疏高维特征的场景。

通过离散扩散生成预训练,可在所有特征字段上同时提供密集重构监督,缓解点击率预测中因数据稀疏导致的表征坍缩问题。然而现有生成式CTR方法存在根本缺陷:重构目标对所有特征字段赋予相等训练权重,忽视了高基数ID字段、稀疏类别属性、数值型特征与行为序列之间重建难度的巨大差异。这导致简单字段主导训练梯度,而最难但最具信息量的字段长期欠拟合,我们称之为生成难度失衡。为此提出HeteGenCTR,通过与去噪网络联合训练的每字段可学习难度参数,实现统一信号驱动下的两个协同组件:自平衡损失自动将梯度预算分配给更难字段,并保证可证明稳定的平衡点;难度引导注意力机制抑制已收敛简单字段的影响,增强跨字段信息流向困难字段。两者共享同一学习信号,全程保持一致性。在五个CTR基准数据集及七天线上A/B测试中,相比当前最优基线均有显著且统计显著的提升,对冷启动与长尾用户收益尤为突出。

原文摘要 · Abstract (English)

Generative pre-training via discrete diffusion provides dense reconstruction supervision across all feature fields simultaneously, mitigating representation collapse from data sparsity in CTR prediction. However, all existing generative CTR methods share a fundamental limitation: the reconstruction objective assigns equal training weight to every feature field, ignoring the profound heterogeneity of reconstruction difficulty across high-cardinality ID fields, sparse categorical attributes, numerical values, and behavioral sequences. This causes easy fields to dominate training gradients while the hardest but most informative fields remain chronically underfit, a problem we term the generative difficulty imbalance.We propose HeteGenCTR, which resolves this imbalance through per-field learnable difficulty parameters jointly trained with the denoising network. This unified signal drives two coordinated components without additional hyperparameters: a self-balancing loss that automatically reallocates gradient budget toward harder fields with a provably stable equilibrium, and a difficulty-guided attention mechanism that suppresses the influence of already-converged easy fields while amplifying cross-field information flow toward hard fields. Both components share the same learned signal and remain mutually consistent throughout training. Experiments on five CTR benchmarks and a seven-day online A/B test demonstrate consistent, statistically significant improvements over state-of-the-art baselines, with disproportionate gains for cold-start and long-tail users.

点击率预测特征生成扩散模型异构建模

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。