用多智能体搜索优化提示词，提升对话系统表现
SAGE: Stochastic Prompt Optimization via Agent-Guided Exploration
- 通过多智能体执行诊断代码，实现提示词空间的随机搜索
- 在三个基准上验证有效，连续优化使留存率显著提升
- 适合需要持续改进的开放任务对话系统开发者
上下文工程已成为不更新参数即可提升AI系统性能的主要手段。近期研究表明文本梯度并非真实梯度，促使我们将自动提示优化（APO）视为黑盒搜索问题。本文提出SPO（Stochastic Prompt Optimization），一种在提示空间进行随机搜索的框架，并对比三种逐步复杂的策略：基于错误信息的随机搜索、带有进化算子的遗传算法，以及SAGE（SPO via Agent-Guided Exploration），即包含诊断代码执行的多智能体流水线。在三个基准上的实验表明，无单一策略始终最优，效果取决于景观结构与错误类型的交互。进一步将SAGE部署于心理健康聊天机器人，在连续优化范式下，将八轮各自噪声较大的A/B测试结果整合，获得统计显著的次日留存率提升。我们认为，结合定性诊断与定量验证，是使智能体优化在开放任务导向对话中有效的关键。
原文摘要 · Abstract (English)
Context engineering has emerged as a primary lever for improving AI systems without parameter updates. Recent work showing that textual gradients do not function as real gradients motivates treating automatic prompt optimization (APO) as black-box search. We introduce SPO (Stochastic Prompt Optimization), a framework for stochastic search over prompt space, and compare three strategies of increasing sophistication: error-informed random search, a genetic algorithm with evolutionary operators, and SAGE (SPO via Agent-Guided Exploration), a multi-agent pipeline with diagnostic code execution. Across three benchmarks, no single strategy dominates; effectiveness depends on the interaction of landscape structure with error type. We further deploy SAGE on a mental-health chatbot under a continuous optimization paradigm, where it compounds eight cycles of individually-noisy A/B tests into a statistically robust gain in next-day retention. We argue that coupling qualitative diagnosis with quantitative validation is what makes agentic optimization effective for open-ended task-oriented dialogue.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。