用交错模块设计融合序列与特征交互,提升推荐系统性能。
Bumblebee: Interleaved Mixed-Layer Building Blocks for Large-Scale Recommendation Systems
- 将序列建模与特征交叉整合为可堆叠的微型流水线单元
- 在多个任务上优于基线模型,且无需增加参数量
- 适合需要高效高精度推荐的工业级系统
推荐系统近年经历了显著演进,从传统特征交互模块转向生成式下一步动作预测。发展主要沿两个方向:序列建模和特征交互。本文提出Bumblebee架构,通过交错可堆叠的模块设计,弥合两者间的缺失交互。每个模块包含序列个性化、基于注意力的编码和特征交叉的微流水线,生成联合表征供下一模块使用。该机制促进早期多次模态混合,增强下游特征的上下文信息。块间残差连接构建跨模态路径,提升预测性能而不增加参数。模块可通过选择性移除组件实现质量与吞吐的灵活权衡。在大规模工业数据上评估显示,其在分类与回归任务中持续优于基线模型。消融实验确认交错组合是性能提升主因。结果表明,交错异构功能单元而非深层堆叠,是未来推荐架构的有前景范式。
原文摘要 · Abstract (English)
Recommendation systems have undergone significant transformations in the past years. The transition from traditional feature interaction modules to generative next-action prediction has pushed the boundaries of personalized content. Developments have largely evolved along two separate tracks. Sequence modeling approaches on the one hand and feature interaction methods on the other. In this paper, we introduce Bumblebee, a recommendation architecture that addresses the lack of interaction between the two directions through an interleaved, stackable block design. Each block implements a micro-pipeline of layers combining sequence personalization, attention-based encoding, and feature crossing into a self-contained unit. Every block produces a joint representation of both feature modalities which is consumed by the next block in the sequence. This mechanism encourages early and repeated mixture of modalities and enriches downstream features with additional contextual information. Residual connections between blocks create cross-modal information pathways and yield additional predictive performance without adding additional parameters. Blocks can be specialized by selectively dropping components, enabling flexible trade-offs between quality and throughput. We evaluate our approach on large-scale industrial data and show consistent improvements over comparable baseline models across several classification and regression tasks. Furthermore, we conduct ablation studies to confirm that the interleaved composition itself is the primary driver of these improvements. Our results suggest that interleaving heterogeneous functional units, rather than composing deep stacks, is a promising paradigm for future-generation recommendation architectures.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。