构建统一评估框架，让化学合成预测模型比得更准、更公平。
Procrustean Bed for AI-Driven Retrosynthesis: A Unified Framework for Reproducible Evaluation
- 将不同模型输出转为统一格式，实现可复现的严格对比
- 发现高成功率模型常含化学无效路径，且难处理长程合成规划
- 适合关注模型真实性能与可复现性的研发人员
计算机辅助合成规划（CASP）的发展因缺乏标准化评估体系和依赖仅关注拓扑完成度的指标而受阻。我们提出RetroCast，一个统一的评估套件，将异构模型输出标准化为统一模式，支持统计严谨的直接比较。该框架包含分层采样和自助置信区间验证的可复现基准测试流程，并配套SynthArena交互式平台用于路线定性审查。我们在新制定的标准基准上评估了主流搜索类与序列类算法。分析显示，'可解性'（库存终止率）与路线质量存在偏差：高可解性分数常掩盖化学无效性，且与实验真实路径重现性无关。此外，我们发现搜索类方法在复杂度陡增时出现性能断崖式下降，远不如序列类方法在长程合成规划中的表现。我们开放完整框架、基准定义及标准化模型预测数据库，推动领域透明化与可复现发展。
原文摘要 · Abstract (English)
Progress in computer-aided synthesis planning (CASP) is obscured by the lack of standardized evaluation infrastructure and the reliance on metrics that prioritize topological completion over chemical validity. We introduce RetroCast, a unified evaluation suite that standardizes heterogeneous model outputs into a common schema to enable statistically rigorous, apples-to-apples comparison. The framework includes a reproducible benchmarking pipeline with stratified sampling and bootstrapped confidence intervals, accompanied by SynthArena, an interactive platform for qualitative route inspection. We utilize this infrastructure to evaluate leading search-based and sequence-based algorithms on a new suite of standardized benchmarks. Our analysis reveals a divergence between "solvability" (stock-termination rate) and route quality; high solvability scores often mask chemical invalidity or fail to correlate with the reproduction of experimental ground truths. Furthermore, we identify a "complexity cliff" in which search-based methods, despite high solvability rates, exhibit a sharp performance decay in reconstructing long-range synthetic plans compared to sequence-based approaches. We release the full framework, benchmark definitions, and a standardized database of model predictions to support transparent and reproducible development in the field.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。