让机器学习实验过程可追溯，自动记录训练细节。
yProv4ML: Effortless Provenance Tracking for Machine Learning Systems
- 用PROV-JSON格式记录训练全过程，兼容性好。
- 仅需少量代码改动即可捕获模型演化轨迹。
- 适合需要可复现性的研究者与工程团队使用。
大规模语言模型（LLMs）的兴起展现了其强大的灵活性与泛化能力，吸引了众多研究者关注。然而，这一发展也暴露出开发过程缺乏透明度和严谨性的问题，尤其是无法提前确定训练轮数等超参数，导致难以识别最优模型。现有工具如MLFlow虽能自动化收集部分信息，但多采用专有格式且忽视数据血缘关系。本文提出yProv4ML框架，以极小的代码修改代价，在PROV-JSON格式下自动捕获机器学习过程中的完整溯源信息，实现对训练过程的可追溯管理。
原文摘要 · Abstract (English)
The rapid growth of interest in large language models (LLMs) reflects their potential for flexibility and generalization, and attracted the attention of a diverse range of researchers. However, the advent of these techniques has also brought to light the lack of transparency and rigor with which development is pursued. In particular, the inability to determine the number of epochs and other hyperparameters in advance presents challenges in identifying the best model. To address this challenge, machine learning frameworks such as MLFlow can automate the collection of this type of information. However, these tools capture data using proprietary formats and pose little attention to lineage. This paper proposes yProv4ML, a framework to capture provenance information generated during machine learning processes in PROV-JSON format, with minimal code modifications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。