解决大规模推荐系统中嵌入碰撞问题,提升模型新鲜度与性能。
Multi-Probe Zero Collision Hash (MPZCH): Mitigating Embedding Collisions and Enhancing Model Freshness in Large-Scale Recommenders
- 采用多探针零碰撞哈希机制,避免传统哈希表的嵌入冲突。
- 线上实验显示用户嵌入零碰撞,物品嵌入新鲜度显著提升。
- 适合需要高精度个性化推荐的生产级推荐系统使用。
嵌入表是大规模推荐系统的核心组件,用于将高基数类别特征高效映射为稠密向量表示。然而,随着唯一ID数量增加,传统基于哈希的索引方法面临嵌入碰撞问题,导致模型性能和个性化质量下降。本文提出多探针零碰撞哈希(MPZCH),一种基于线性探测的新型索引机制,能有效缓解嵌入碰撞。在合理表尺寸下,通常可完全消除碰撞,同时保持生产规模效率。MPZCH利用辅助张量和高性能CUDA核实现可配置的探测与主动淘汰策略,通过淘汰过时ID并重置重新分配槽位,防止哈希方法常见的陈旧嵌入继承,确保新特征能从零开始学习。尽管存在碰撞缓解开销,其训练QPS和推理延迟仍与现有方法相当。严格的线上实验表明,MPZCH实现了用户嵌入的零碰撞,并显著提升物品嵌入的新鲜度与质量。该方案已开源至TorchRec库,供社区使用。
原文摘要 · Abstract (English)
Embedding tables are critical components of large-scale recommendation systems, facilitating the efficient mapping of high-cardinality categorical features into dense vector representations. However, as the volume of unique IDs expands, traditional hash-based indexing methods suffer from collisions that degrade model performance and personalization quality. We present Multi-Probe Zero Collision Hash (MPZCH), a novel indexing mechanism based on linear probing that effectively mitigates embedding collisions. With reasonable table sizing, it often eliminates these collisions entirely while maintaining production-scale efficiency. MPZCH utilizes auxiliary tensors and high-performance CUDA kernels to implement configurable probing and active eviction policies. By retiring obsolete IDs and resetting reassigned slots, MPZCH prevents the stale embedding inheritance typical of hash-based methods, ensuring new features learn effectively from scratch. Despite its collision-mitigation overhead, the system maintains training QPS and inference latency comparable to existing methods. Rigorous online experiments demonstrate that MPZCH achieves zero collisions for user embeddings and significantly improves item embedding freshness and quality. The solution has been released within the open-source TorchRec library for the broader community.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。