解决稀疏文本图中图文对齐失效问题,提升模型迁移能力。
S2Aligner: Pair-Efficient and Transferable Pre-Training for Sparse Text-Attributed Graphs

- 分离语义与结构表征,用拓扑信号增强对齐而不污染语义空间。
- 在低文本覆盖率场景下仍保持高精度,跨域性能超越基线模型。
- 适合处理真实世界中文本稀疏的图数据,如社交网络、知识图谱。
在文本属性图(TAGs)上进行预训练是构建可迁移图基础模型的核心。当前主流方法依赖大语言模型作为对齐器,通过语义知识将图与文本表征对齐,但通常假设节点文本提供充分且可靠的监督信号,这一假设在真实世界的稀疏标签图中常被打破。当文本锚点缺失、噪声多或跨域分布不均时,图结构需与弱语义证据对齐,导致结构-语义对应不可靠,并引发稀疏性带来的迁移偏差。本文提出S2Aligner,一种针对稀疏文本属性图的感知稀疏性、结构增强的LLM-as-Aligner框架。其核心思想是解耦语义对齐与结构建模,使拓扑感知信号能增强对齐而不污染共享语义空间。具体地,将图-文表示分解为语义与结构分量,采用一致性控制的结构重建注入可靠拓扑线索至文本表示,并在文本稀疏时抑制不一致结构信号。此外,引入稀疏性感知的跨域风险平衡机制,通过全局域密度比校准域风险,基于图可靠性估计降低不可靠稀疏样本权重。理论分析表明该目标可减少跨域泛化差距。在多种图领域、不同稀疏度和下游任务上的大量实验表明,S2Aligner持续优于现有基线。
原文摘要 · Abstract (English)
Pre-training on text-attributed graphs (TAGs) is central to building transferable graph foundation models, where LLM-as-Aligner methods align graph and text representations through the semantic knowledge of large language models. However, these methods usually assume that node texts provide sufficient and reliable supervision, an assumption often violated in real-world sparse TAGs. When textual anchors are missing, noisy, or uneven across domains, graph structures must be aligned with weak semantic evidence, leading to unreliable structure-semantics correspondence and sparsity-induced transfer bias. This paper presents S2Aligner, a sparsity-aware and structure-enhanced LLM-as-Aligner framework for graph-text pre-training on sparse TAGs. The key idea is to decouple semantic alignment from structural modeling, allowing topology-aware signals to enhance alignment without contaminating the shared semantic space. Specifically, S2Aligner decomposes graph-text representations into semantic and structural components, uses structure-oriented reconstruction with consistency control to inject reliable topology cues into text representations, and suppresses inconsistent structural signals under textual sparsity. Moreover, S2Aligner introduces sparsity-aware cross-domain risk balancing, which calibrates domain risks through a global-domain density ratio and downweights unreliable sparse samples via graph reliability estimation. Theoretical analysis shows that this objective reduces cross-domain generalization gaps by controlling domain risk discrepancy. Extensive experiments across diverse graph domains, sparsity levels, and downstream tasks demonstrate that S2Aligner consistently outperforms existing baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。