无需训练即可用大模型直接分析多表数据库,开箱即用。
No Need to Train Your RDB Foundation Model
- 限定在统一单位的列内压缩数据,避免跨列混淆
- 不依赖参数训练,仍能保持模型表达能力
- 适合作为现有大模型的数据库前处理工具
关系型数据库(RDB)包含大量异构表格数据,可用于预测建模。但在企业场景中目标众多,如何避免每次新增预测任务都需重新训练模型?基于上下文学习(ICL)的基座模型提供了一种便捷方案,但目前主要局限于单表操作。要扩展到多张相互关联的表,关键在于将可变大小的数据库邻域压缩为固定长度的ICL输入。我们通过理论与实证证明:应仅在具有统一单位和角色的高维列内进行压缩,而非跨列混合不同数据类型——后者需大量标注信息才能判断相关性。在此约束下,我们发现编码器的表达能力不受无参数设计影响。由此提出一套无需训练的、可无缝对接现有单表ICL基座模型的数据库编码器体系。实际应用中,我们开发了可扩展的SQL原语实现该编码器,构建出开源的RDBLearn基座模型,能在未见过的数据集上即开即用,表现稳健。
原文摘要 · Abstract (English)
Relational databases (RDBs) contain vast amounts of heterogeneous tabular information that can be exploited for predictive modeling purposes. But since the space of potential targets is vast across enterprise settings, how can we avoid retraining a new model each time we wish to predict a new quantity of interest? Foundation models based on in-context learning (ICL) offer a convenient option, but so far are largely restricted to single-table operability. In generalizing to multiple interrelated tables, it is essential to compress variably-sized RDB neighborhoods into fixed-length ICL samples for consumption by the decoder. However, the details here are critical: unlike existing supervised learning RDB pipelines, we provide theoretical and empirical evidence that ICL-specific compression should be constrained within high-dimensional RDB columns where all entities share units and roles, not across columns where the relevance of heterogeneous data types cannot be determined without extensive label information. Conditioned on this restriction, we then demonstrate that encoder expressiveness is actually not compromised by excluding trainable parameters. Hence we arrive at a principled family of RDB encoders that can be seamlessly paired with already-existing single-table ICL foundation models, whereby no training or fine-tuning is required. From a practical standpoint, we develop scalable SQL primitives to implement the encoder stage, resulting in the easy-to-use open-source RDBLearn foundation model capable of robust performance on unseen datasets out of the box.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。