arXiv:2410.09119cs.LGcs.IR2024-10被引 8

lucie能自动导入95%的UCI机器学习数据集,解决原工具无法读取的问题。

$\textit{lucie}$: An Improved Python Package for Loading Datasets from the UCI Machine Learning Repository

  • 自动识别非标准格式,无需手动处理即可导入数据
  • 在130个数据集上成功率达95.4%,远超原工具的73.1%
  • 专为高人气数据集设计,适合科研与教学快速调用数据

加州大学欧文分校(UCI)机器学习存储库(UCIMLR)是被广泛引用的数据集资源,包含数百个高影响力数据集。然而,其中28.4%的前250名热门数据集无法通过官网推荐的ucimlrepo包导入,因它们以.zip形式存放且采用非标准格式,需额外手动处理。为此,本文提出lucie——一个自动判断数据格式并导入大量此前不可用数据集的工具,尽可能保持表格结构。lucie基于前100个最流行数据集设计,并在后续130个数据集上测试,成功率达95.4%,显著优于ucimlrepo的73.1%。lucie已作为PyPI上的Python包发布,代码覆盖率高达98%。

原文摘要 · Abstract (English)

The University of California--Irvine (UCI) Machine Learning (ML) Repository (UCIMLR) is consistently cited as one of the most popular dataset repositories, hosting hundreds of high-impact datasets. However, a significant portion, including 28.4% of the top 250, cannot be imported via the $\textit{ucimlrepo}$ package that is provided and recommended by the UCIMLR website. Instead, they are hosted as .zip files, containing nonstandard formats that are difficult to import without additional ad hoc processing. To address this issue, here we present $\textit{lucie}$ -- $\underline{l}oad$ $\underline{U}niversity$ $\underline{C}alifornia$ $\underline{I}rvine$ $\underline{e}xamples$ -- a utility that automatically determines the data format and imports many of these previously non-importable datasets, while preserving as much of a tabular data structure as possible. $\textit{lucie}$ was designed using the top 100 most popular datasets and benchmarked on the next 130, where it resulted in a success rate of 95.4% vs. 73.1% for $\textit{ucimlrepo}$. $\textit{lucie}$ is available as a Python package on PyPI with 98% code coverage.

数据集Python工具UCI数据集自动化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。