CCFormer通过高效跨场交互与分层压缩,提升工业推荐系统性能与效率。
CCFormer: Efficient Cross-Field Interaction and Hierarchical Sequence Compression for Industrial Recommendation at Tencent

- 采用分离字段的交叉注意力与子空间令牌混合,捕捉跨域长期偏好
- 分层序列压缩使长序列建模更高效,信息损失减少
- 在腾讯视频与广告场景中显著提效,线上收益提升1.7%-3.57%
近年来,工业推荐系统中的自注意力模型可通过延长序列和扩大容量受益于可预测的缩放规律。然而,实际系统对延迟和资源有严格限制,难以平衡计算开销与细粒度特征交互。本文提出CCFormer,一种统一跨场特征交互与长序列压缩的高效Transformer骨干网络。CCFormer结合特征字段分离的交叉注意力与长序列子空间令牌混合,以挖掘异构特征域间的长期偏好信号;采用逐级扩展感受野的分层序列压缩策略,实现高效长序列建模并降低信息丢失。在两个公开基准和一个大规模工业数据集上的实验表明,CCFormer持续优于现有先进基线。在腾讯的视频推荐与广告排序场景的在线A/B测试中,分别带来3.57%的点击率提升和1.71%的广告收入增长,同时训练速度比强基线HSTU快2.21倍。CCFormer已全面部署于腾讯生产推荐系统,服务两大场景主流量。
原文摘要 · Abstract (English)
Recent studies in industrial recommendation systems have demonstrated that sequential recommendation models built upon self-attention can benefit from predictable scaling laws by increasing sequence length and model capacity. However, practical recommender systems impose strict latency and resource constraints, making it challenging to balance computational overhead with fine-grained feature interaction. In this paper, we propose CCFormer, an efficient Transformer backbone that unifies cross-field feature interaction and compressed long-sequence modeling for industrial recommendation. Specifically, CCFormer combines feature-field separated cross attention with long-sequence subspace token mixing to exploit long-term preference signals across heterogeneous feature domains. A hierarchical sequence compression strategy with progressively expanded receptive fields enables efficient long-sequence modeling with reduced information loss. Extensive experiments on two public benchmarks and a large-scale industrial dataset demonstrate that CCFormer consistently outperforms state-of-the-art baselines. Online A/B tests in a video recommendation scenario and an advertising ranking scenario at Tencent further validate its industrial practicality, yielding a 3.57% CTR gain and a 1.71% advertising revenue lift, respectively, while accelerating model training by 2.21x over the strong HSTU baseline. CCFormer has been fully deployed in Tencent's production recommendation system, serving the main traffic of both scenarios.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。