用大模型模拟专家调参，量化控制算法的优化潜力。
Crucible: Quantifying the Potential of Control Algorithms through LLM Agents
- 用大模型构建多层级专家模拟器，自动探索算法调参空间。
- 在多种控制任务中验证，不同算法的可调优范围差异显著。
- 适合算法设计者和系统工程师参考，提升实际部署性能。
生产环境中的控制算法通常需要领域专家针对具体场景调整参数与逻辑，但现有研究主要关注理想或默认配置下的算法性能，忽视了关键的调优潜力。为弥补这一缺口，我们提出Crucible，一个基于大模型驱动的多层级专家模拟系统，用于评估算法的调优潜力，并定义了一个形式化指标进行量化分析。我们在从经典控制任务到复杂计算机系统的广泛案例研究中验证了Crucible的有效性，并在真实部署中确认其结论。实验结果表明，Crucible能够系统性地量化不同算法的可调空间。此外，该方法为算法分析与设计提供了新维度，最终带来性能提升。代码已开源：https://github.com/thu-media/Crucible。
原文摘要 · Abstract (English)
Control algorithms in production environments typically require domain experts to tune their parameters and logic for specific scenarios. However, existing research predominantly focuses on algorithmic performance under ideal or default configurations, overlooking the critical aspect of Tuning Potential. To bridge this gap, we introduce Crucible, an agent that employs an LLM-driven, multi-level expert simulation to turn algorithms and defines a formalized metric to quantitatively evaluate their Tuning Potential. We demonstrate Crucible's effectiveness across a wide spectrum of case studies, from classic control tasks to complex computer systems, and validate its findings in a real-world deployment. Our experimental results reveal that Crucible systematically quantifies the tunable space across different algorithms. Furthermore, Crucible provides a new dimension for algorithm analysis and design, which ultimately leads to performance improvements. Our code is available at https://github.com/thu-media/Crucible.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。