解决图注意力模型中消息过量聚合导致的信息稀释问题
Relieving the Over-Aggregating Effect in Graph Transformers
- 将节点消息聚合拆分为并行过程，限制单次聚合输入量
- 通过排序加权机制优先保留关键信息，减少信息丢失
- 无需修改主模型，可直接提升现有图注意力性能
图注意力在图学习任务中表现优异，但全局交互学习因节点数量庞大而面临挑战。本文发现一种新现象——过聚合：大量消息汇聚至单一节点，缺乏区分度，导致关键信息被稀释甚至丢失。为此，提出Wideformer，一种即插即用的图注意力改进方法。该方法将所有节点的聚合操作分解为并行流程，限制每次聚合的输入规模，避免消息稀释；同时通过排序与加权步骤，引导模型关注更具信息量的聚合输出。实验表明，Wideformer能有效缓解过聚合问题，使基础模型更聚焦于关键信息，显著优于基线方法。
原文摘要 · Abstract (English)
Graph attention has demonstrated superior performance in graph learning tasks. However, learning from global interactions can be challenging due to the large number of nodes. In this paper, we discover a new phenomenon termed over-aggregating. Over-aggregating arises when a large volume of messages is aggregated into a single node with less discrimination, leading to the dilution of the key messages and potential information loss. To address this, we propose Wideformer, a plug-and-play method for graph attention. Wideformer divides the aggregation of all nodes into parallel processes and guides the model to focus on specific subsets of these processes. The division can limit the input volume per aggregation, avoiding message dilution and reducing information loss. The guiding step sorts and weights the aggregation outputs, prioritizing the informative messages. Evaluations show that Wideformer can effectively mitigate over-aggregating. As a result, the backbone methods can focus on the informative messages, achieving superior performance compared to baseline methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。