用AI自动修复过时的机器学习笔记，让代码重跑成功
Automated Modernization of Machine Learning Engineering Notebooks for Reproducibility
- 用大模型+自动执行反馈，逐条修复代码错误和运行问题
- 在8210个无法运行的笔记中，使40%-45%恢复可复现
- 适合需要重用旧代码的研究者和工程师
交互式计算笔记本（如Jupyter）广泛用于机器学习工程中，从数据准备到模型训练与评估的全流程编程与共享。然而，由于硬件与软件生态的快速演进，许多已发表的机器学习工程笔记在当前环境中已无法复现，阻碍了代码重用与科学进展。我们分析了75个热门Kaggle竞赛中选取的12,106个笔记：仅26%仍可复现。关键发现是，回滚环境依赖（即降级依赖以匹配提交时间）反而使可复现性降至12%，并引入更多失败模式。为此，我们设计并实现了MLEModernizer——一个基于大模型的代理框架，将当前环境视为固定约束，主动现代化笔记代码以恢复可复现性。MLEModernizer通过迭代执行、收集执行反馈，并应用三类针对性修复：错误修复、运行时优化和得分校准。在8,210个基线环境不可复现的笔记上评估，MLEModernizer使3,292个（40.1%，GPT-5.2）和3,683个（44.9%，GPT-OSS-120b）笔记恢复可复现。该方法提供一种尽力而为的自动化恢复与现代化技术，能提升部分笔记的可复现性。实践者可借助MLEModernizer验证、重用和维护随软硬件演进而不断变化的机器学习成果。
原文摘要 · Abstract (English)
Interactive computational notebooks (e.g., Jupyter notebooks) are widely used in machine learning engineering (MLE) to program and share end-to-end pipelines, from data preparation to model training and evaluation. However, environmental erosion-the rapid evolution of hardware and software ecosystems for machine learning-has rendered many published MLE notebooks non-reproducible in contemporary environments, hindering code reuse and scientific progress. To quantify this gap, we study 12,106 notebooks selected from 75 popular Kaggle competitions: only 26% remain reproducible today. Crucially, we find that environment backporting, i.e., downgrading dependencies to match the submission time, does not improve reproducibility (decreased to 12%) but rather introduces additional failure modes. To address environmental erosion, we design and implement MLEModernizer, an LLM-driven agentic framework that treats the contemporary environment as a fixed constraint and modernizes notebook code to restore reproducibility. MLEModernizer iteratively executes notebooks, collects execution feedback, and applies three types of targeted fixes: error-repair, runtime-reduction, and score-calibration. Evaluated on 8,210 notebooks that are non-reproducible under the baseline environment, MLEModernizer makes 3,292 (40.1%, GPT-5.2) and 3,683 (44.9%, GPT-OSS-120b) notebooks reproducible. MLEModernizer presents a best-effort automated recovery and modernization technique that can improve reproducibility for a subset of notebooks. Practitioners can leverage MLEModernizer to validate, reuse, and maintain MLE artifacts as the hardware and software ecosystems continue to evolve.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。