让稀疏特征延迟参与交互,提升点击率预测精度
Deferred is Better: A Framework for Multi-Granularity Deferred Interaction of Heterogeneous Features
- 按信息密度分组特征,分阶段引入交互过程
- 延迟引入低信息特征,避免干扰高信息特征信号
- 适合特征稀疏性差异大的推荐系统场景
点击率(CTR)预测模型通过建模海量特征空间中的交互来估计用户-物品点击概率。一个常被忽视的根本挑战是特征固有的异质性:其稀疏性和信息量差异巨大。例如,物品ID等类别特征极度稀疏,而物品价格等数值特征相对密集。现有CTR模型大多忽略此问题,对所有特征采用统一的交互策略,同时输入交互层。这导致低信息特征过早引入,产生显著噪声并掩盖高信息特征信号,引发模型崩溃,阻碍鲁棒表示的学习。为此,我们提出多粒度信息感知延迟交互网络(MGDIN),自适应地推迟特征进入交互过程。其核心机制分为两步:首先,采用多粒度特征分组策略,将原始特征划分为信息密度更均一的组别,缓解极端稀疏特征的影响,并支持从多视角捕捉特征交互;其次,通过分层掩码策略实现延迟交互,控制每组特征在何时、以何种方式参与,早期掩码低信息组,随网络加深逐步解封。这种延迟引入使模型先基于高信息特征建立稳健理解,再渐进融合其他组的稀疏信息。
原文摘要 · Abstract (English)
Click-through rate (CTR) prediction models estimates the probability of a user-item click by modeling interactions across a vast feature space. A fundamental yet often overlooked challenge is the inherent heterogeneity of these features: their sparsity and information content vary dramatically. For instance, categorical features like item IDs are extremely sparse, whereas numerical features like item price are relatively dense. Prevailing CTR models have largely ignored this heterogeneity, employing a uniform feature interaction strategy that inputs all features into the interaction layers simultaneously. This approach is suboptimal, as the premature introduction of low-information features can inject significant noise and mask the signals from information-rich features, which leads to model collapse and hinders the learning of robust representations. To address the above challenge, we propose a Multi-Granularity Information-Aware Deferred Interaction Network (MGDIN), which adaptively defers the introduction of features into the feature interaction process. MGDIN's core mechanism operates in two stages: First, it employs a multi-granularity feature grouping strategy to partition the raw features into distinct groups with more homogeneous information density in different granularities, thereby mitigating the effects of extreme individual feature sparsity and enabling the model to capture feature interactions from diverse perspectives. Second, a delayed interaction mechanism is implemented through a hierarchical masking strategy, which governs when and how each group participates by masking low-information groups in the early layers and progressively unmasking them as the network deepens. This deferred introduction allows the model to establish a robust understanding based on high-information features before gradually incorporating sparser information from other groups...
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。