通过用户与物品的协同聚类,实现推荐系统嵌入表压缩,大幅降低参数量且不损失精度。
Balanced Co-Clustering of Users and Items for Embedding Table Compression in Recommender Systems
- 基于用户-物品交互的协同信号进行双向聚类,让相似实体共享嵌入。
- 压缩率超75%,召回率下降不超过1.85%,训练速度最快提升346倍。
- 适合资源受限场景下的工业级推荐系统部署。
过去十年,推荐系统通过深度学习将每个用户/物品映射为稠密嵌入向量取得了显著进展。在工业规模下,这些用户/物品的嵌入表包含海量参数,在训练和推理时带来沉重的计算与内存负担,制约了模型在资源受限环境中的部署。现有嵌入压缩方法或严重损害推荐精度,或引入显著计算开销。本文提出BACO框架,一种快速高效的嵌入表压缩方案。不同于传统ID哈希,BACO利用用户-物品交互中的协同信号对用户和物品进行分组,使相似实体共享代码本中的同一嵌入。我们构建了一个平衡共聚类目标,最大化簇内连通性同时约束簇体积均衡,并通过严格的理论分析将经典图聚类技术统一整合进该框架。为获得有效分组并避免代码本坍塌,BACO采用合理的权重设计、高效标签传播求解器及次级用户簇机制。在多个基准数据集上的大量实验表明,相较于完整模型和18个基线,BACO可实现超过75%的嵌入参数压缩,召回率下降最多仅1.85%,且速度比最强基线快达346倍。
原文摘要 · Abstract (English)
Recommender systems have advanced markedly over the past decade by transforming each user/item into a dense embedding vector with deep learning models. At industrial scale, embedding tables constituted by such vectors of all users/items demand a vast amount of parameters and impose heavy compute and memory overhead during training and inference, hindering model deployment under resource constraints. Existing solutions towards embedding compression either suffer from severely compromised recommendation accuracy or incur considerable computational costs. To mitigate these issues, this paper presents BACO, a fast and effective framework for compressing embedding tables. Unlike traditional ID hashing, BACO is built on the idea of exploiting collaborative signals in user-item interactions for user and item groupings, such that similar users/items share the same embeddings in the codebook. Specifically, we formulate a balanced co-clustering objective that maximizes intra-cluster connectivity while enforcing cluster-volume balance, and unify canonical graph clustering techniques into the framework through rigorous theoretical analyses. To produce effective groupings while averting codebook collapse, BACO instantiates this framework with a principled weighting scheme for users and items, an efficient label propagation solver, as well as secondary user clusters. Our extensive experiments comparing BACO against full models and 18 baselines over benchmark datasets demonstrate that BACO cuts embedding parameters by over 75% with a drop of at most 1.85% in recall, while surpassing the strongest baselines by being up to 346X faster.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。