用分块注意力提升推荐系统点击率，工业级部署效果显著
HHFT: Hierarchical Heterogeneous Feature Transformer for Recommendation Systems
- 按语义分组异构特征，避免不同类型信息混淆
- 引入专用投影与全连接层，提升特征交互精度
- 在淘宝平台实现点击率提升0.4%，GMV增长0.6%
我们提出HHFT（Hierarchical Heterogeneous Feature Transformer），一种面向工业级点击率预测的Transformer架构。针对传统DNN的局限性，设计三个关键机制：(1) 语义特征分块：将用户画像、商品信息、行为序列等异构特征按语义划分为独立模块，保留领域特异性信息；(2) 异构Transformer编码器：采用块内专用的QKV投影与前馈网络，避免不同类型特征间的语义混淆；(3) Hiformer层：捕捉跨特征的高阶交互关系。实验表明，该模型显著优于DNN基线，在大规模场景下实现CTR AUC提升0.4%。已在淘宝生产平台成功部署，带来关键业务指标显著提升，其中成交总额（GMV）增长0.6%。
原文摘要 · Abstract (English)
We propose HHFT (Hierarchical Heterogeneous Feature Transformer), a Transformer-based architecture tailored for industrial CTR prediction. HHFT addresses the limitations of DNN through three key designs: (1) Semantic Feature Partitioning: Grouping heterogeneous features (e.g. user profile, item information, behaviour sequennce) into semantically coherent blocks to preserve domain-specific information; (2) Heterogeneous Transformer Encoder: Adopting block-specific QKV projections and FFNs to avoid semantic confusion between distinct feature types; (3) Hiformer Layer: Capturing high-order interactions across features. Our findings reveal that Transformers significantly outperform DNN baselines, achieving a +0.4% improvement in CTR AUC at scale. We have successfully deployed the model on Taobao's production platform, observing a significant uplift in key business metrics, including a +0.6% increase in Gross Merchandise Value (GMV).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。