针对图推荐系统提出动态量化方法,提升低精度部署下的推荐效果。
A Node-Aware Dynamic Quantization Approach for Graph Collaborative Filtering
- 根据节点在图中的结构关系动态调整量化尺度,避免固定量化导致误差累积。
- 2比特量化下召回率提升27.8%,归一化折扣累计收益提高17.6%。
- 适合边缘设备部署,模型压缩8-12倍且训练速度翻倍,适合实际应用。
在协同过滤推荐系统中,图神经网络(GNN)虽表现优异,但因嵌入参数量大、计算成本高,在资源受限的边缘设备上部署困难。直接对节点嵌入进行常规量化会忽略其图结构特性,导致消息传递过程中的误差累积,降低量化后嵌入质量。为此,我们提出面向协同过滤的图结构感知动态量化训练方法(GNAQ),通过融合图交互关系,自适应地为每个节点嵌入设定量化尺度。具体而言,基于节点特征分布初始化量化区间,并在GNN层的消息传递过程中动态优化。该方法缓解了固定量化尺度带来的信息损失,捕捉用户-物品交互图中的层次语义特征。此外,采用图关系感知的梯度估计替代传统直通估计器,保障训练时梯度传播更准确。在四个真实数据集上的实验表明,GNAQ在2比特量化下相比当前最优方法(BiGeaR和N2UQ)平均提升27.8% Recall@10与17.6% NDCG@10;尤其可实现全精度模型性能保持的同时,模型尺寸缩小8至12倍,且训练时间比基线方法快一倍。
原文摘要 · Abstract (English)
In the realm of collaborative filtering recommendation systems, Graph Neural Networks (GNNs) have demonstrated remarkable performance but face significant challenges in deployment on resource-constrained edge devices due to their high embedding parameter requirements and computational costs. Using common quantization method directly on node embeddings may overlooks their graph based structure, causing error accumulation during message passing and degrading the quality of quantized embeddings.To address this, we propose Graph based Node-Aware Dynamic Quantization training for collaborative filtering (GNAQ), a novel quantization approach that leverages graph structural information to enhance the balance between efficiency and accuracy of GNNs for Top-K recommendation. GNAQ introduces a node-aware dynamic quantization strategy that adapts quantization scales to individual node embeddings by incorporating graph interaction relationships. Specifically, it initializes quantization intervals based on node-wise feature distributions and dynamically refines them through message passing in GNN layers. This approach mitigates information loss caused by fixed quantization scales and captures hierarchical semantic features in user-item interaction graphs. Additionally, GNAQ employs graph relation-aware gradient estimation to replace traditional straight-through estimators, ensuring more accurate gradient propagation during training. Extensive experiments on four real-world datasets demonstrate that GNAQ outperforms state-of-the-art quantization methods, including BiGeaR and N2UQ, by achieving average improvement in 27.8\% Recall@10 and 17.6\% NDCG@10 under 2-bit quantization. In particular, GNAQ is capable of maintaining the performance of full-precision models while reducing their model sizes by 8 to 12 times; in addition, the training time is twice as fast compared to quantization baseline methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。