构建首个统一的气动外形优化评估基准,支持跨任务公平比较。
ShapeBench: A Scalable Benchmark and Diagnostic Suite for Standardized Evaluation in Aerodynamic Shape Optimization

- 设计103个任务覆盖8类形状,含快速代理模型与高保真仿真验证
- 不同任务中优化器排名差异大,平均相关性仅0.013,结论难泛化
- 适合对比传统方法与基于大模型的新方法,推动通用优化算法发展
气动外形优化(ASO)快速发展,但缺乏标准化评估框架。为实现公平比较,需统一基准涵盖多样形状类别、目标函数形式及匹配预算的先进基线。本文提出ShapeBench,一个开源的ASO基准,提供统一API,覆盖8类形状的103个任务和多种优化范式。每个任务配备经验证的快速代理模型以支持高效搜索;在可行情况下,还提供高保真计算流体动力学(CFD)流水线用于最终验证,支持系统性精度差距分析。ShapeBench提供可复现协议和配置良好的基线,使用一致预算度量进行公平比较,适用于传统优化器与基于大模型的方法,包括通用优化器和新提出的领域专用进化型大模型基线ShapeEvolve。ShapeBench结果表明,优化器在不同形状类别和问题设置下的排名存在显著差异,平均成对斯皮尔曼等级相关系数ρ=0.013,说明单任务结论无法可靠推广至其他问题类型。此外,当前基准尚未饱和:传统方法极少能在所有形状类别和任务上适用,凸显了对更通用方法的需求。
原文摘要 · Abstract (English)
Rapid progress in aerodynamic shape optimization (ASO) has outpaced currently-available standardized evaluation frameworks. Fair comparison requires a unified benchmark spanning diverse shape classes, objective formulations, and matched-budget state-of-the-art baselines. We introduce ShapeBench, an open-source ASO benchmark with a unified API spanning 103 tasks across eight shape categories and multiple optimization regimes. Each ShapeBench task includes a validated surrogate for fast search; when feasible, a high-fidelity Computational Fluid Dynamics (CFD) pipeline for final verification is available, enabling systematic fidelity-gap analysis. ShapeBench provides a reproducible protocol with well-configured baselines to compare fairly using a consistent budget metric, allowing for comparison among both classical and LLM-driven methods, including general-purpose optimizers and a new domain-specialized evolutionary LLM baseline, ShapeEvolve. Results on ShapeBench demonstrate substantial variance in optimizer rankings across shape categories and problem formulations, with mean pairwise Spearman $ρ= 0.013$, so single-task conclusions do not reliably generalize across problem classes. The benchmark is also far from saturation; classical methods are rarely applicable across all shape categories and tasks, further highlighting the need for more general-purpose approaches.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。