用博弈论方法高效剪枝推荐系统嵌入参数,节省存储且无需重训练。
On-device Content-based Recommendation with Single-shot Embedding Pruning: A Cooperative Game Perspective
- 基于博弈论的贡献度评估,用谢尔比值量化嵌入参数价值。
- 单次计算即可适配多种参数预算,性能优于现有方法。
- 适合资源受限设备上的推荐系统部署,如移动端和联邦学习场景。
基于内容的推荐系统在电子商务、在线广告和个性化推荐中至关重要。然而,由于存在大量类别特征,嵌入表在实际部署中面临严重存储瓶颈,尤其在资源受限设备上。现有嵌入剪枝方法大多需为每个目标参数预算进行昂贵的重训练,导致巨大计算开销。在现实中,这成为联邦学习和流式场景等多样化存储需求下的主要障碍。本文提出谢尔比值引导的嵌入压缩方法(Shaver),从合作博弈视角出发,利用谢尔比值量化每个嵌入参数的贡献,实现基于贡献的参数剪枝。针对谢尔比值固有的高计算成本,我们提出一种高效无偏估计方法。此外,在剪枝阶段引入字段感知码本,缓解传统置零操作带来的信息损失。在三个真实世界数据集上的大量实验表明,Shaver 在不同参数预算下均表现出与轻量级推荐模型相当的性能。源代码已开源:https://github.com/chenxing1999/shaver。
原文摘要 · Abstract (English)
Content-based Recommender Systems (CRSs) play a crucial role in shaping user experiences in e-commerce, online advertising, and personalized recommendations. However, due to the vast amount of categorical features, the embedding tables used in CRS models pose a significant storage bottleneck for real-world deployment, especially on resource-constrained devices. To address this problem, various embedding pruning methods have been proposed, but most existing ones require expensive retraining steps for each target parameter budget, leading to enormous computation costs. In reality, this computation cost is a major hurdle in real-world applications with diverse storage requirements, such as federated learning and streaming settings. In this paper, we propose Shapley Value-guided Embedding Reduction (Shaver) as our response. With Shaver, we view the problem from a cooperative game perspective, and quantify each embedding parameter's contribution with Shapley values to facilitate contribution-based parameter pruning. To address the inherently high computation costs of Shapley values, we propose an efficient and unbiased method to estimate Shapley values of a CRS's embedding parameters. Moreover, in the pruning stage, we put forward a field-aware codebook to mitigate the information loss in the traditional zero-out treatment. Through extensive experiments on three real-world datasets, Shaver has demonstrated competitive performance with lightweight recommendation models across various parameter budgets. The source code is available at https://github.com/chenxing1999/shaver
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。