用知识图谱融合实验数据,提升机器学习元学习的预测与相似性分析。
Integrating Meta-Features with Knowledge Graph Embeddings for Meta-Learning
- 构建统一知识图谱,整合数据集与算法管道的实验记录。
- 在14.4万条OpenML实验上实现高精度性能预测与相似数据集检索。
- 开源模型与基准数据集,推动元学习研究标准化。
网络上海量的机器学习实验记录为元学习提供了巨大机遇,可通过历史实验提升新任务表现。核心任务包括:管道性能估计(PPE),即预测算法在目标数据集上的表现;以及基于数据集性能的相似性估计(DPSE),用于识别具有相似性能模式的数据集。现有方法主要依赖数据集元特征(如样本数、类别熵等)进行数值表示,但忽略了大量过往实验结果和管道元数据,难以捕捉数据集与算法间的交互关系,限制了对性能相似性的建模能力。本文提出KGmetaSP,一种基于知识图嵌入的方法,利用已有实验数据建模这种交互关系,以提升PPE与DPSE性能。我们将数据集与算法管道统一建模于知识图谱(KG)中,生成支持泛化元模型的嵌入向量(用于PPE)及支持距离检索的嵌入(用于DPSE)。为验证方法,我们构建了一个包含144,177条OpenML实验的大规模基准,支持跨数据集评估。实验表明,KGmetaSP仅需一个泛化元模型即可实现精准性能预测,并显著优于基线方法。所提方法、知识图谱与基准已开源,为元学习设立了新标准,展示了将开放实验数据整合至统一知识图谱对领域发展的推动作用。
原文摘要 · Abstract (English)
The vast collection of machine learning records available on the web presents a significant opportunity for meta-learning, where past experiments are leveraged to improve performance. Two crucial meta-learning tasks are pipeline performance estimation (PPE), which predicts pipeline performance on target datasets, and dataset performance-based similarity estimation (DPSE), which identifies datasets with similar performance patterns. Existing approaches primarily rely on dataset meta-features (e.g., number of instances, class entropy, etc.) to represent datasets numerically and approximate these meta-learning tasks. However, these approaches often overlook the wealth of past experimental results and pipeline metadata available. This limits their ability to capture dataset - pipeline interactions that reveal performance similarity patterns. In this work, we propose KGmetaSP, a knowledge-graph-embeddings approach that leverages existing experiment data to capture these interactions and improve both PPE and DPSE. We represent datasets and pipelines within a unified knowledge graph (KG) and derive embeddings that support pipeline-agnostic meta-models for PPE and distance-based retrieval for DPSE. To validate our approach, we construct a large-scale benchmark comprising 144,177 OpenML experiments, enabling a rich cross-dataset evaluation. KGmetaSP enables accurate PPE using a single pipeline-agnostic meta-model and improves DPSE over baselines. The proposed KGmetaSP, KG, and benchmark are released, establishing a new reference point for meta-learning and demonstrating how consolidating open experiment data into a unified KG advances the field.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。