融合Transformer与GNN优势，实现网页搜索排序的统一建模
Pre-trained Graphformer-based Ranking at Web-scale Search (Extended Abstract)
- 采用模块化胶囊结构预训练，统一处理查询-网页对与图结构
- 在大规模搜索数据上验证，显著优于传统单一模型方法
- 适合从事搜索引擎排序优化的研究者与工程师参考
Transformer与图神经网络（GNN）已被用于学习排序（LTR）任务。前者基于查询-网页对进行排序分值回归，后者则在查询-网页二分图中进行链接预测，两者虽互补但存在分布差异。尽管可在源数据上预训练GNN或Transformer，再在标注稀疏的LTR数据上微调，但二者在配对与图结构域间的分布偏移使融合困难。为此，本文提出MPGraf模型，采用模块化与胶囊式预训练策略，旨在协同整合Transformer的回归能力与GNN的链接预测优势。通过大规模离线与在线实验，全面评估了MPGraf性能。
原文摘要 · Abstract (English)
Both Transformer and Graph Neural Networks (GNNs) have been employed in the domain of learning to rank (LTR). However, these approaches adhere to two distinct yet complementary problem formulations: ranking score regression based on query-webpage pairs, and link prediction within query-webpage bipartite graphs, respectively. While it is possible to pre-train GNNs or Transformers on source datasets and subsequently fine-tune them on sparsely annotated LTR datasets, the distributional shifts between the pair-based and bipartite graph domains present significant challenges in integrating these heterogeneous models into a unified LTR framework at web scale. To address this, we introduce the novel MPGraf model, which leverages a modular and capsule-based pre-training strategy, aiming to cohesively integrate the regression capabilities of Transformers with the link prediction strengths of GNNs. We conduct extensive offline and online experiments to rigorously evaluate the performance of MPGraf.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。