解决图数据向量量化中的代码本坍缩问题,提升表示能力与泛化性。
Graph is a Natural Regularization: Revisiting Vector Quantization for Graph Representation Learning
- 引入图结构与特征相似性作为正则信号,增强代码本利用。
- 采用Gumbel-Softmax实现软分配,确保所有码字都有梯度更新。
- 适用于需要高表达力图表示的下游任务,如节点分类、链接预测。
向量量化(VQ)近年来成为学习图结构数据压缩离散表示的有前景方法。然而,代码本坍缩这一基础挑战在图领域仍缺乏深入研究,严重限制了图标记的表达力与泛化能力。本文通过实证研究发现,在图重构任务中,即使采用视觉或语言领域提出的缓解策略,联合训练VQ与图神经网络时代码本坍缩仍持续发生。从数据与优化角度诊断表明,坍缩与特征冗余、连通密度等图数据特性相关,并受确定性硬分配训练动态强化。为此,提出RGVQ框架,显式融合图拓扑与特征相似性作为正则信号,提升代码本利用率并促进标记多样性。RGVQ通过Gumbel-Softmax实现软分配,确保所有码字接收梯度更新;同时引入结构感知对比正则,惩罚将相同标记分配给不相似节点对的行为。大量实验表明,RGVQ显著提升代码本利用率,一致增强现有先进图VQ骨干模型在多个下游任务上的性能,实现更富表达力与可迁移的图标记表示。
原文摘要 · Abstract (English)
Vector Quantization (VQ) has recently emerged as a promising approach for learning compressed and discrete representations for graph-structured data. However, a fundamental challenge, i.e., codebook collapse, remains underexplored in the graph domain, significantly limiting the expressiveness and generalization of graph tokens.In this paper, we present an empirical study and observe that codebook collapse consistently occurs when training VQ jointly with Graph Neural Networks under graph reconstruction tasks, even with mitigation strategies proposed in vision or language domains. Moreover, we provide a diagnosis of collapse from data and optimization perspectives, showing that collapse is associated with graph data properties such as feature redundancy and connectivity density, and is further reinforced by the training dynamics of deterministic hard assignment. To address these issues, we propose RGVQ, a novel framework that integrates graph topology and feature similarity as explicit regularization signals to enhance codebook utilization and promote token diversity. RGVQ introduces soft assignments via Gumbel-Softmax reparameterization, ensuring that all codewords receive gradient updates. In addition, RGVQ incorporates a structure-aware contrastive regularization to penalize assigning the same token to dissimilar node pairs. Extensive experiments demonstrate that RGVQ substantially improves codebook utilization and consistently boosts the performance of state-of-the-art graph VQ backbones across multiple downstream tasks, enabling more expressive and transferable graph token representations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。