用标准化元数据让机器自动复现模型实验，告别手动调试。
Croissant Tasks: A Metadata Format for Reproducible Machine Learning Evaluations
- 设计可被机器理解的元数据格式，分离任务与实现细节。
- 能自动将现有基准测试转为该格式，生成可运行的复现流程。
- 适合追求自动化复现的科研人员和开源项目维护者。
可复现性是科学方法的核心，但在机器学习中仍面临严峻挑战，主要源于执行细节不明确和软件环境脆弱。人工补救措施如检查清单和手动验证虽有帮助，但耗时且难以扩展。为此，我们提出 Croissant Tasks：一种声明式、可机器执行的元数据格式，将底层实现细节抽象为高层规范。该格式支持概念性复现：通过独立的智能体生成实现，而非复制脆弱的源代码。我们贡献了三点：(1) Croissant Tasks 规范，形式化解耦任务问题与解决方案；(2) 自动化 LLM 流水线，将现有基准测试转化为该格式；(3) 实证验证表明，自主智能体可基于这些规范从零生成功能完整且准确的复现流程。我们期望该格式成为机器学习中自动化与概念性复现的新基石。
原文摘要 · Abstract (English)
Reproducibility is fundamental to the scientific method, yet remains a critical challenge in machine learning. Contributing factors include underspecified execution details and brittle software environments. Human-centric remedies, such as checklists and manual verification, help but require intensive effort and fail to scale. To address this, we introduce Croissant Tasks: a declarative, machine-actionable metadata format that abstracts low-level implementation details into high-level specifications. This format enables conceptual reproducibility: verifying claims via independent, agent-generated implementations rather than brittle source code replication. We contribute: (1) the Croissant Tasks specification, formally decoupling task problem from solution; (2) an automated LLM pipeline that retrofits existing benchmarks into this format; and (3) empirical validation showing autonomous agents can ingest these specifications to generate functional, accurate reproduction pipelines from scratch. We envision this format as a new foundation for automated and conceptual reproducibility in machine learning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。