构建了覆盖9408种机器学习流程的综合性数据集,解决算法选择中实验成本高、代表性不足的问题。
PIPES: A Meta-dataset of Machine Learning Pipelines
- 系统生成9408种组合式机器学习流程,涵盖多种数据预处理与模型策略
- 在300个数据集上完成实验,包含训练测试时间、预测结果和错误信息等完整记录
- 适用于算法选择、元学习研究者,尤其适合需要多样化流程评估的场景
机器学习中的算法选择问题(ASP)面临高计算成本,因需在给定数据集上评估多种算法性能。为降低此成本,元学习领域可利用OpenML等在线仓库中的历史实验数据。但分析发现,OpenML存在流程多样性不足的问题,尤其在数据预处理环节(如归一化、缺失值填补)的覆盖有限,且各模块常集中于少数热门技术,导致样本分布不均。为此,本文提出PIPES,一个涵盖多种流程组合的实验数据集,旨在实现多样性和完整性。PIPES包含对300个数据集应用9,408种不同机器学习流程的实验结果,记录了各流程模块详情、训练与测试时间、预测输出、性能指标及错误信息。该数据集支持跨多样化流程与数据集的深入分析,具备可扩展性,未来可集成更多数据与实验。数据、代码、补充材料及全部实验均可在https://github.com/cynthiamaia/PIPES.git获取。
原文摘要 · Abstract (English)
Solutions to the Algorithm Selection Problem (ASP) in machine learning face the challenge of high computational costs associated with evaluating various algorithms' performances on a given dataset. To mitigate this cost, the meta-learning field can leverage previously executed experiments shared in online repositories such as OpenML. OpenML provides an extensive collection of machine learning experiments. However, an analysis of OpenML's records reveals limitations. It lacks diversity in pipelines, specifically when exploring data preprocessing steps/blocks, such as scaling or imputation, resulting in limited representation. Its experiments are often focused on a few popular techniques within each pipeline block, leading to an imbalanced sample. To overcome the observed limitations of OpenML, we propose PIPES, a collection of experiments involving multiple pipelines designed to represent all combinations of the selected sets of techniques, aiming at diversity and completeness. PIPES stores the results of experiments performed applying 9,408 pipelines to 300 datasets. It includes detailed information on the pipeline blocks, training and testing times, predictions, performances, and the eventual error messages. This comprehensive collection of results allows researchers to perform analyses across diverse and representative pipelines and datasets. PIPES also offers potential for expansion, as additional data and experiments can be incorporated to support the meta-learning community further. The data, code, supplementary material, and all experiments can be found at https://github.com/cynthiamaia/PIPES.git.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。