arXiv:2608.08189cs.AI2026-08

让大模型同时进化程序和评估器,用更少真实测试完成更好设计。

Janus: An Algorithm-Evaluator Co-Evolution Framework for LLM-Driven Discovery under Expensive Evaluation Budgets

论文配图:Janus: An Algorithm-Evaluator Co-Evolution Framework for LLM-Driven Discovery under Expensive Evaluation Budgets
图 1 · 摘自论文原文
  • 大模型协同演化目标程序与可执行评估器,动态适应搜索偏差。
  • 平均仅用59.1%的真实评估次数,达到基准99%的优化效果。
  • 适合高成本实验场景,如物理模拟、硬件验证等科学工程任务。

基于大模型的程序发现依赖快速评估反馈,但许多科学与工程任务需高保真仿真、硬件执行或物理实验,导致每次评估成本高昂。廉价替代评估器虽能降低成本,但固定代理易受搜索引发的数据分布偏移影响,且在稀疏、搜索偏倚的标签下难以可靠拟合。我们提出Janus框架,利用大模型协同演化目标程序与可执行代理评估器。为缓解标签稀缺,Janus借助大模型中的领域知识生成特定任务的评估器程序,并通过真实结果校准。为应对分布偏移,Janus同步演化评估器,采用促进对齐目标选择,并以在线信用更新维护区域条件下的评估器组合。由于代理预测仍可能出错,Janus仅用其优先排序候选者,要求真实验证后才允许进入目标程序种群或更新最优解。在五个科学与工程设计任务中,Janus在真实评估预算内实现更大的最佳当前改进曲线下面积,最终性能优于仅演化目标程序的基线。平均仅使用59.1%的真实评估次数,即达成基线99%的最终改进。演化后的代理评估器也比初始版本更准确地识别有前景候选。这些成果将评估引导的大模型发现从低成本可扩展任务拓展至评估稀缺且昂贵的科学领域。

原文摘要 · Abstract (English)

LLM-driven program discovery relies on rapid evaluator feedback, but many scientific and engineering tasks require high-fidelity simulations, hardware execution, or physical experiments, making each evaluation expensive. Cheap surrogate evaluators can reduce this cost, yet fixed surrogates are vulnerable to search-induced distribution shift and are difficult to fit reliably from sparse, search-biased labels. We introduce Janus, a framework that uses LLMs to co-evolve target programs and executable proxy evaluators. To address label scarcity, Janus leverages domain knowledge encoded in LLMs to generate task-specific evaluator programs and calibrates them using real outcomes. To mitigate distribution shift, Janus evolves evaluators alongside target programs, selects them using a promotion-aligned objective, and maintains region-conditioned portfolios with online credit updates. Because proxy predictions remain fallible, Janus uses them only to prioritize candidates and requires real validation before candidates can enter the target-program population or update the incumbent. Across five scientific and engineering design tasks, Janus achieves a larger area under the best-so-far improvement curve over the real-evaluation budget and higher final performance than a matched baseline that evolves only target programs. On average, Janus reaches 99/% of the baseline's final improvement with 59.1/% fewer real evaluations. Evolved proxy evaluators also rank promising candidates more accurately than their seed versions. Together, these results extend evaluator-guided LLM discovery from tasks with cheap, scalable feedback to scientific domains where trustworthy evaluation is scarce and expensive.

大模型程序发现评估器演化科学计算

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。