通过重连图结构与量化嵌入,显著降低推荐系统存储与计算开销。
Lightweight Embeddings with Graph Rewiring for Collaborative Filtering
- 用量化和可选参与机制压缩嵌入存储,提升资源受限设备的部署能力。
- 预训练后通过无梯度整数规划剪枝低贡献节点,构建稀疏重连图降低计算量。
- 在真实工业数据集上实现更高精度,同时存储和推理成本大幅下降。
随着推荐服务快速扩展,其部署常面临资源受限边缘设备的挑战。基于图神经网络的推荐系统存在高嵌入存储成本和图传播带来的运行时延迟问题,尤其涉及大量乘累加操作(MACs)。此前工作LEGCF虽有效降低存储成本,但在更严格的存储限制下性能下降,且未解决图传播带来的计算开销。为此,我们提出轻量级嵌入重连图方法(LERG),在保留LEGCF组合式码本结构基础上,引入量化技术进一步压缩存储,支持更多元嵌入。在资源丰富的服务器上预训练量化嵌入表后,采用无梯度二元整数规划进行微调,识别并剪枝低贡献实体,构建剔除这些节点的重连图。最终将量化嵌入表与稀疏重连图迁移至边缘设备,显著降低计算内存与推理时间。在三个公开基准数据集(含一个工业级数据集)上的实验表明,LERG在保持优异推荐性能的同时,大幅减少图推荐服务的存储与计算成本。
原文摘要 · Abstract (English)
As recommendation services scale rapidly and their deployment now commonly involves resource-constrained edge devices, GNN-based recommender systems face significant challenges, including high embedding storage costs and runtime latency from graph propagations. Our previous work, LEGCF, effectively reduced embedding storage costs but struggled to maintain recommendation performance under stricter storage limits. Additionally, LEGCF did not address the extensive runtime computation costs associated with graph propagation, which involves heavy multiplication and accumulation operations (MACs). These challenges consequently hinder effective training and inference on resource-constrained edge devices. To address these limitations, we propose Lightweight Embeddings with Rewired Graph for Graph Collaborative Filtering (LERG), an improved extension of LEGCF. LERG retains LEGCFs compositional codebook structure but introduces quantization techniques to reduce the storage cost, enabling the inclusion of more meta-embeddings within the same storage. To optimize graph propagation, we pretrain the quantized compositional embedding table using the full interaction graph on resource-rich servers, after which a fine-tuning stage is engaged to identify and prune low-contribution entities via a gradient-free binary integer programming approach, constructing a rewired graph that excludes these entities (i.e., user/item nodes) from propagating signals. The quantized compositional embedding table with selective embedding participation and sparse rewired graph are transferred to edge devices which significantly reduce computation memory and inference time. Experiments on three public benchmark datasets, including an industry-scale dataset, demonstrate that LERG achieves superior recommendation performance while dramatically reducing storage and computation costs for graph-based recommendation services.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。