提出基于连通性度量的新型位置编码，让Transformer更好理解图结构。
Communicability-Inspired Positional Encoding (CIPE)
- 用路径连通性构建位置编码，使注意力机制能直接感知节点间关系
- 在7个基准上平均提升35.5%，显著优于现有方法
- 适合需要理解复杂图结构的模型，尤其对结构敏感任务有帮助
位置编码（PEs）对Transformer至关重要。然而，在非欧几里得图上设计有效的位置编码仍具挑战性。理想的位置编码应构建一种与注意力兼容的几何结构：不仅描述图结构，还能使内积反映有意义的结构相关性。为此，我们提出基于连通性的位置编码（CIPE），其基于连通性度量——一个聚合所有长度路径贡献的节点对度量。通过构造，CIPE的内积可恢复连通性，将全局多路径连通性转化为可被注意力使用的相似性几何。为实际训练，我们引入维度对齐机制，将依赖图大小的CIPE表示映射到预设维度，同时忠实保留诱导几何。实验表明，CIPE在七个基准上平均提升35.5%，显著优于代表性位置编码；同时在结构感知型图Transformer中也持续表现更优，而其他编码常仅带来微弱改进。这些结果确立了CIPE作为注意力兼容图位置编码的原理性框架。
原文摘要 · Abstract (English)
Positional encodings (PEs) are essential for Transformers. Yet designing effective PEs for non-Euclidean graphs remains challenging. Such encodings should ideally induce an Attention-Compatible Geometry for self-attention: not merely describing graph structure, but defining a geometry whose inner products reflect meaningful structural relatedness. To realize this geometry, we propose Communicability-Inspired Positional Encoding (CIPE), built from communicability, a measure between pairs of nodes that aggregates contributions from paths of all lengths. By construction, CIPE inner products recover communicability, converting global multi-path connectivity into an attention-ready similarity geometry. For practical Transformer training, we introduce dimensionality alignment, mapping graph-size-dependent CIPE representations to prescribed dimensions while faithfully preserving the induced geometry. Empirically, CIPE improves structure-agnostic Transformers by 35.5% on average across seven benchmarks, outperforming representative PEs; it also consistently improves structure-biased graph Transformers, where competing PEs often yield only marginal benefits. These results position CIPE as a principled framework for attention-compatible graph positional encodings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。