让机器学习代理提前预测方案优劣，大幅加速科学发现。
Can We Predict Before Executing Machine Learning Agents?
- 用预训练模型基于数据报告预测方案好坏，替代耗时执行。
- 在1.8万组对比中预测准确率达61.5%，且置信度可靠。
- 新框架使收敛速度提升6倍，适合需快速试错的研究场景。
自主机器学习代理正推动科学发现变革，但受限于生成-执行-反馈的范式。以往方法因依赖昂贵的物理执行进行假设评估，存在严重执行瓶颈。为突破此限制，我们内化执行先验，以即时预测推理替代耗时运行检查，借鉴世界模型思想。本文正式定义了以数据为中心的方案偏好预测任务，并构建包含18,438对比较的综合语料库。实验表明，当输入经验证的数据分析报告时，大语言模型（LLMs）展现出显著预测能力，达到61.5%的准确率，并具备稳健的置信度校准。我们在此基础上构建了FOREAGENT，采用预测-验证循环机制，在保证性能的同时实现6倍收敛加速，超越基于执行的基线+6%。代码与数据集已公开于https://github.com/zjunlp/predict-before-execute。
原文摘要 · Abstract (English)
Autonomous machine learning agents have revolutionized scientific discovery, yet they remain constrained by a Generate-Execute-Feedback paradigm. Previous approaches suffer from a severe Execution Bottleneck, as hypothesis evaluation relies strictly on expensive physical execution. To bypass these physical constraints, we internalize execution priors to substitute costly runtime checks with instantaneous predictive reasoning, drawing inspiration from World Models. In this work, we formalize the task of Data-centric Solution Preference and construct a comprehensive corpus of 18,438 pairwise comparisons. We demonstrate that LLMs exhibit significant predictive capabilities when primed with a Verified Data Analysis Report, achieving 61.5% accuracy and robust confidence calibration. Finally, we instantiate this framework in FOREAGENT, an agent that employs a Predict-then-Verify loop, achieving a 6x acceleration in convergence while surpassing execution-based baselines by +6%. Our code and dataset are publicly available at https://github.com/zjunlp/predict-before-execute.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。