让复杂AI系统各组件协同优化，局部奖励与全局性能对齐。
Optimas: Optimizing Compound AI Systems with Globally Aligned Local Rewards
- 为每个组件设计局部奖励函数，确保局部优化推动全局提升。
- 在五个真实系统上平均提升11.92%，显著优于基线方法。
- 适用于大模型、工具、传统模型混合的复杂系统优化。
集成大型语言模型、专用工具和传统机器学习模型的复合人工智能系统正被广泛应用于解决复杂的现实任务。然而，由于系统结构不可微且各组件配置类型多样（包括提示词、超参数和模型参数），优化复合系统仍具挑战性。为此，我们提出Optimas——一种统一的复合系统优化框架。核心思想是为每个组件维护一个局部奖励函数（LRF），满足局部-全局对齐属性：即组件的局部奖励与系统整体性能相关。每轮迭代中，Optimas高效调整各LRF以保持该属性，同时最大化各组件的局部奖励。该方法允许使用指定优化方式独立更新异构配置，且确保局部改进始终带来整体性能提升。我们在五个真实复合系统上进行了广泛评估，结果表明Optimas相比强基线平均提升11.92%，提供了一种通用且高效的复合系统优化方案。
原文摘要 · Abstract (English)
Compound AI systems integrating multiple components, such as Large Language Models, specialized tools, and traditional machine learning models, are increasingly deployed to solve complex real-world tasks. However, optimizing compound systems remains challenging due to their non-differentiable structures and diverse configuration types across components, including prompts, hyperparameters, and model parameters. To address this challenge, we propose Optimas, a unified framework for effective optimization of compound systems. The core idea of Optimas is to maintain one Local Reward Function (LRF) per component, each satisfying a local-global alignment property, i.e., each component's local reward correlates with the global system performance. In each iteration, Optimas efficiently adapts the LRFs to maintain this property while simultaneously maximizing each component's local reward. This approach enables independent updates of heterogeneous configurations using the designated optimization method, while ensuring that local improvements consistently lead to performance gains. We present extensive evaluations across five real-world compound systems to demonstrate that Optimas outperforms strong baselines by an average improvement of 11.92%, offering a general and effective approach for improving compound systems. Our website is at https://optimas.stanford.edu.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。