用多标记池化提升图结构信息编码，让LLM更好回答图问答问题。
Is One Token All It Takes? Graph Pooling Tokens for LLM-based GraphQA
- 通过多标记池化扩大图到语言模型的信息通道
- 结合LoRA使虚拟节点池化等方法达到73%的准确率
- 适合研究图神经网络与大模型融合的学者
将图神经网络（GNN）与大语言模型（LLM）结合是图问答（GraphQA）的前沿方向，但如何有效将复杂结构信息编码进LLM隐空间仍是挑战。现有先进架构如G-Retriever通常依赖标准GNN和激进的均值池化，将整个图子结构压缩为单一令牌，造成严重信息瓶颈。本文提出两种互补策略：（1）通过多令牌池化增加图到LLM接口的带宽；（2）通过全局注意力机制提升图编码器语义质量。评估了包括Top-k、SAGPool、DiffPool、MinCutPool和虚拟节点池化（VNPool）在内的多种分层剪枝与聚类池化算子，将图数据投影为多个可学习令牌。实验证明，尽管池化在软提示微调中引入显著不稳定性，但低秩适应（LoRA）能有效稳定特定分层投影（特别是VNPool和剪枝方法），而密集聚类算子仍具挑战性。该稳定化使压缩表示媲美全图基线（在WebQSP上达~73% Hit@1）。概念上，我们证明采用VNPool的图变压器在结构上等同于单层Perceiver IO编码器。最后，我们将FandE（特征与边）评分适配至生成式图问答领域。分析揭示图问答基准存在表征饱和问题，目标答案常与孤立节点特征高度相关。代码已开源。
原文摘要 · Abstract (English)
The integration of Graph Neural Networks (GNNs) with Large Language Models (LLMs) has emerged as a promising paradigm for Graph Question Answering (GraphQA). However, effective methods for encoding complex structural information into the LLM's latent space remain an open challenge. Current state-of-the-art architectures, such as G-Retriever, typically rely on standard GNNs and aggressive mean pooling to compress entire graph substructures into a single token, creating a severe information bottleneck. This work mitigates this bottleneck by investigating two orthogonal strategies: (1) increasing the bandwidth of the graph-to-LLM interface via multi-token pooling, and (2) enhancing the semantic quality of the graph encoder via global attention mechanisms. We evaluate a suite of hierarchical pruning and clustering-based pooling operators including Top-k, SAGPool, DiffPool, MinCutPool, and Virtual Node Pooling (VNPool) to project graph data into multiple learnable tokens. Empirically, we demonstrate that while pooling introduces significant instability during soft prompt tuning, the application of Low-Rank Adaptation (LoRA) effectively stabilizes specific hierarchical projections (notably VNPool and pruning methods), though dense clustering operators remain challenging. This stabilization allows compressed representations to rival full-graph baselines (achieving ~73% Hit@1 on WebQSP). Conceptually, we demonstrate that a Graph Transformer with VNPool implementation functions structurally as a single-layer Perceiver IO encoder. Finally, we adapt the FandE (Features and Edges) Score to the generative GraphQA domain. Our analysis reveals that the GraphQA benchmark suffers from representational saturation, where target answers are often highly correlated with isolated node features. The implementation is available at https://github.com/Agrover112/G-Retriever/tree/all_good/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。