用智能体+统一框架,让难复现的工业健康预测论文变可比基准。
From paper to benchmark: agentic, framework-based reproduction of under-specified methods in machine health intelligence

- 通过结构化接口将论文描述转为可执行组件
- 16篇论文复现成功率显著提升,结果可跨论文对比
- 适合工业界和学术界做模型评估与方法比较
工业故障预测与健康管理(PHM)是应用机器学习中一个典型难题:如何将发表论文转化为可运行、可评测的实现。由于工业数据访问受限、预处理与评估协议不完整,以及隐含设计选择(如滑动窗口、目标构建、数据划分)的存在,现有方法复现困难。现有纸到代码系统生成的实现因假设不一致而难以比较。本文提出基于智能体与共享框架的PHM论文复现方法,通过槽位绑定接口将公式与协议描述映射为任务定义、数据适配器、窗口化、目标、模型与评估器等结构化组件,并显式记录未明确假设。生成结果经由标准化任务契约与评估钩子验证,确保一致性与可比性。我们在16篇PHM论文上评估该方法,对比框架增强型、技能驱动型与提示驱动型智能体与近期无框架的纸到代码代理。结果表明,结合智能体与共享框架能将论文复现从孤立代码生成转变为可执行、假设感知且系统可比的基准实现。
原文摘要 · Abstract (English)
Industrial Prognostics and Health Management (PHM) provides a representative case study for a broader challenge in applied machine learning: translating published papers into executable, benchmark-ready implementations. Reproducing under-specified methods in PHM is particularly difficult due to restricted access to industrial datasets, incomplete reporting of preprocessing and evaluation protocols, and implicit design choices (e.g., windowing, target construction, data splits) that critically affect performance. Existing paper-to-code systems generate implementations for individual papers, but these artifacts are often not directly comparable due to inconsistencies in assumptions and evaluation settings. We introduce \emph{agentic, framework-based PHM paper reproduction}, where an agent translates a paper into a shared PHM benchmark framework via a \emph{slot-binding interface}. This interface maps equations and protocol descriptions into structured components (task definitions, dataset adapters, windowing, targets, models, and evaluators), while explicitly recording unresolved assumptions. The resulting implementations are validated against standardized task contracts and evaluation hooks, enabling consistent and comparable benchmarking. We evaluate this approach on 16 PHM papers, comparing framework-enhanced, skill-based and prompt-based agentic reproduction against a recent framework-free paper-reproduction agent. We assess reproduction success, model-based code evaluation, framework binding of paper assumptions, and cross-paper benchmark comparability under standardized protocols. Our results show that coupling agentic generation with a shared framework transforms paper reproduction from isolated code synthesis into executable, assumption-aware, and systematically comparable benchmark implementations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。