通过轻量提示优化量化图神经网络聚合，提升低比特模型性能
LoRAP: Low-Rank Aggregation Prompting for Quantized Graph Neural Networks Training
- 在图聚合层注入低秩提示，动态优化量化后特征
- 9个数据集上4种框架均实现性能提升，最低损失仅0.5%
- 适合资源受限场景下的高效图模型部署
图神经网络（GNN）通过消息传递机制处理图数据，捕捉节点间的关联。为在资源受限环境下降低模型规模并加速推理，图神经网络量化成为重要方向。与大语言模型量化相比，图特征量化在GNN中更为关键。受此启发，我们提出利用提示学习（prompt learning）操纵输入数据，以改进图神经网络量化感知训练（QAT）的性能。针对仅对节点特征进行提示时，仅能优化部分量化聚合结果的问题，我们引入低秩聚合提示（LoRAP），在每个聚合特征中注入轻量、依赖输入的提示，以优化量化聚合结果。在9个图数据集上，使用4种主流量化框架的广泛评估表明，LoRAP在引入极小计算开销的前提下，持续提升低比特量化GNN的性能。
原文摘要 · Abstract (English)
Graph Neural Networks (GNNs) are neural networks that aim to process graph data, capturing the relationships and interactions between nodes using the message-passing mechanism. GNN quantization has emerged as a promising approach for reducing model size and accelerating inference in resource-constrained environments. Compared to quantization in LLMs, quantizing graph features is more emphasized in GNNs. Inspired by the above, we propose to leverage prompt learning, which manipulates the input data, to improve the performance of quantization-aware training (QAT) for GNNs. To mitigate the issue that prompting the node features alone can only make part of the quantized aggregation result optimal, we introduce Low-Rank Aggregation Prompting (LoRAP), which injects lightweight, input-dependent prompts into each aggregated feature to optimize the results of quantized aggregations. Extensive evaluations on 4 leading QAT frameworks over 9 graph datasets demonstrate that LoRAP consistently enhances the performance of low-bit quantized GNNs while introducing a minimal computational overhead.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。