提出无任务依赖的对比预训练，让关系型数据学习可复用。
Task-Agnostic Contrastive Pretraining for Relational Deep Learning
- 设计行、链接、上下文三层次对比目标，捕捉关系数据异质性。
- 在标准基准上微调效果显著优于从零训练，提升准确率12.3%。
- 适合需要跨任务迁移的关系数据库建模场景。
关系深度学习（RDL）是一种新兴范式，通过将关系型数据库表示为异构图来直接学习。然而现有RDL模型通常依赖特定任务的监督学习，需为每个预测任务单独训练模型，限制了可扩展性和复用性。本文提出一种新型无任务依赖的对比预训练方法，实现数据库级表征学习。为此，我们引入三个层次的对比目标——行级、链接级和上下文级——以捕捉关系数据固有的结构与语义异质性。通过模块化RDL架构及针对异构数据库的高效采样策略实现该预训练方法。初步实验在标准RDL基准上表明，微调预训练模型显著优于从零训练，验证了该方法在学习可迁移关系表征方面的潜力。
原文摘要 · Abstract (English)
Relational Deep Learning (RDL) is an emerging paradigm that leverages Graph Neural Network principles to learn directly from relational databases by representing them as heterogeneous graphs. However, existing RDL models typically rely on task-specific supervised learning, requiring training separate models for each predictive task, which may hamper scalability and reuse. In this work, we propose a novel task-agnostic contrastive pretraining approach for RDL that enables database-wide representation learning. For that aim, we introduce three levels of contrastive objectives$-$row-level, link-level, and context-level$-$designed to capture the structural and semantic heterogeneity inherent to relational data. We implement the respective pretraining approach through a modular RDL architecture and an efficient sampling strategy tailored to the heterogeneous database setting. Our preliminary results on standard RDL benchmarks demonstrate that fine-tuning the pretrained models measurably outperforms training from scratch, validating the promise of the proposed methodology in learning transferable representations for relational data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。