arXiv:2510.12369cs.IR2025-10

提出分层量化图表示学习框架,实现跨任务自适应的图数据编码。

A Hierarchical Quantized Tokenization Framework for Task-Adaptive Graph Representation Learning

  • 分层量化生成多尺度离散码,支持局部与全局结构信息保留。
  • 在节点分类和链接预测任务上,同等算力下优于现有量化基线。
  • 支持任务自适应路由与双流融合,适合多场景图学习应用。

语言与视觉领域的基础模型得益于统一的离散标记接口,可将原始输入转换为序列以支持大规模预训练与推理。对于图数据,有效的分词器应生成能捕捉节点语义与多尺度关系结构的可复用离散代码,但以往基于量化的图分词器通常采用固定规则组合残差向量量化(RVQ)层级,且仅关注单一结构视图,限制了跨任务迁移能力。本文提出一种具有任务条件路由与双视角标记流的分层量化分词框架。该框架生成多尺度代码及两条同步序列:局部流保留节点级信息,扩散式多跳流总结连通性。轻量级路由器学习任务相关的RVQ深度混合策略,选择合适粒度;门控交叉注意力模块在不修改下游编码器的前提下对齐并融合两路序列。在节点分类与链接预测任务上的实验表明,在相同计算量下性能持续优于强量化基线,消融实验验证了分层量化、自适应路由与融合机制的有效性。

原文摘要 · Abstract (English)

Foundation models in language and vision benefit from a unified discrete token interface that converts raw inputs into sequences for scalable pre-training and inference. For graphs, an effective tokenizer should yield reusable discrete codes that capture both node semantics and relational structure across scales, yet prior quantization-based graph tokenizers typically combine residual vector quantization (RVQ) levels with fixed rules and often focus on a single structural view, limiting cross-task transfer. We present a hierarchical quantized tokenization framework with task-conditioned routing and dual-view token streams. It produces multi-scale codes and two synchronized sequences: a local stream that preserves node-level information and a diffusion-style multi-hop stream that summarizes connectivity. A lightweight router learns task-dependent mixtures over RVQ depths to select an appropriate granularity, while a gated cross-attention module aligns and fuses the two streams into a single token sequence without altering the downstream backbone encoder. Experiments on node classification and link prediction show consistent gains over strong quantized baselines at matched compute, with ablations verifying contributions from hierarchical quantization, adaptive routing, and fusion.

图学习量化自适应序列建模

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。