arXiv:2608.11022cs.DCcs.AI2026-08中稿 · eScience2026

用工作流卡片记录执行细节,让模型和人更懂算法运行全过程。

Workflow Cards: Structured Summaries of Workflow Executions Using Provenance Data

论文配图:Workflow Cards: Structured Summaries of Workflow Executions Using Provenance Data
图 1 · 摘自论文原文
  • 基于执行数据设计结构化卡片模板,捕捉数据处理与参数选择等动态信息。
  • 相比传统查询方式,卡片使大模型回答准确率几乎翻倍。
  • 适合关注可复现性、公平性和系统透明度的研究者与工程师。

Model Cards 和 Data Cards 已证明结构化文档对机器学习成果的价值,涵盖上下文、参数、局限性和预期用途。然而这些实践仍聚焦于静态产物(数据集与训练模型),忽视了生成、转换和评估它们的工作流执行过程。此类执行包含数据预处理、参数选择、运行行为、资源使用及中间变换等关键细节,正是偏差、性能波动与可复现性问题的根源所在。为此,我们提出 Workflow Cards:一种将可机读的工作流溯源数据浓缩为人类与大语言模型(LLMs)均可读、可分析的结构化摘要。本文分两部分:首先,基于从执行级数据中提炼出的代表性溯源问题,定义了 Workflow Card 模板;其次,评估了 LLM 使用卡片理解执行过程的成效,对比基于模式的数据库查询接口。结果表明,Workflow Cards 提供了现有卡片类型(如 Model Cards、Data Cards)所缺失的执行级信息,填补了重要文档空白;且在不同评测方式下,其回答质量接近翻倍。

原文摘要 · Abstract (English)

Model Cards and Data Cards have demonstrated the value of structured, human-readable documentation for machine learning artifacts, capturing their context, parameters, limitations, and intended use. However, these practices remain focused on static artifacts (the datasets and trained models themselves) while overlooking the workflow executions that produce, transform, and evaluate them. Such executions hold critical details about data preparation, parameter choice, runtime behavior, resource use, and intermediate transformations, precisely where bias, performance variation, and reproducibility gaps tend to originate. To close this gap, we introduce Workflow Cards: structured summaries that condense the machine-readable provenance data of a workflow execution into a form both humans and large language models (LLMs) can read and analyze. This paper has two main parts. First, it defines a Workflow Card template informed by a representative set of provenance questions that surface from the execution-level data missing from Model and Data Cards. Second, it evaluates how effectively LLMs use Workflow Cards to understand workflow executions compared with querying provenance databases through a schema-based interface. Results show that Workflow Cards provide execution-level information absent from existing card types, such as Model Cards and Data Cards, thereby filling an important documentation gap; and that Workflow Cards nearly double answer quality compared with schema-based querying, consistently across LLM-as-a-Judge and human assessments.

可复现性工作流溯源数据大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。