用自动调试替代传统优化,让AI agent在无标注数据下快速变聪明。
ROAD: Reflective Optimization via Automated Debugging for Zero-Shot Agent Alignment
- 把失败日志当线索,三智能体协作找出问题根因并生成修复策略
- 仅3轮迭代就提升成功率5.6个百分点,复杂任务性能提高19%
- 适合没有标注数据的冷启动场景,尤其适合实际工程中快速部署LLM代理
自动提示优化(APO)已成为提升大语言模型性能的关键技术,但现有先进方法通常依赖大量标注好的开发数据集来计算进化或强化学习中的适应度分数。然而在真实软件工程中,代理开发初期往往缺乏此类精心构建的数据集,工程师面对的是混乱的生产日志和不断变化的故障模式。本文提出ROAD(基于自动调试的反射式优化),一种新框架,通过将优化过程视为动态调试调查而非随机搜索,避免了对精炼数据集的需求。不同于传统变异策略,ROAD采用专用多智能体架构,包括分析器(Analyzer)用于根因分析、优化器(Optimizer)用于模式聚合、教练(Coach)用于策略整合,将非结构化失败日志转化为稳健的结构化决策树协议。我们在标准学术基准和一个真实生产知识管理系统上评估了ROAD。实验结果表明,ROAD具有极高的样本效率:仅三次自动化迭代即实现成功率提升5.6%(从73.6%增至79.2%),搜索准确率提升3.8%;在零售领域的复杂推理任务中,相比基线,代理性能提升约19%。这些发现表明,模拟人类工程师的故障分析与修复循环,为部署可靠的大语言模型代理提供了一种可行且数据高效的替代方案,无需依赖资源密集型的强化学习训练。
原文摘要 · Abstract (English)
Automatic Prompt Optimization (APO) has emerged as a critical technique for enhancing Large Language Model (LLM) performance, yet current state-of-the-art methods typically rely on large, labeled gold-standard development sets to compute fitness scores for evolutionary or Reinforcement Learning (RL) approaches. In real-world software engineering, however, such curated datasets are rarely available during the initial cold start of agent development, where engineers instead face messy production logs and evolving failure modes. We present ROAD (Reflective Optimization via Automated Debugging), a novel framework that bypasses the need for refined datasets by treating optimization as a dynamic debugging investigation rather than a stochastic search. Unlike traditional mutation strategies, ROAD utilizes a specialized multi-agent architecture, comprising an Analyzer for root-cause analysis, an Optimizer for pattern aggregation, and a Coach for strategy integration, to convert unstructured failure logs into robust, structured Decision Tree Protocols. We evaluated ROAD across both a standardized academic benchmark and a live production Knowledge Management engine. Experimental results demonstrate that ROAD is highly sample-efficient, achieving a 5.6 percent increase in success rate (73.6 percent to 79.2 percent) and a 3.8 percent increase in search accuracy within just three automated iterations. Furthermore, on complex reasoning tasks in the retail domain, ROAD improved agent performance by approximately 19 percent relative to the baseline. These findings suggest that mimicking the human engineering loop of failure analysis and patching offers a viable, data-efficient alternative to resource-intensive RL training for deploying reliable LLM agents.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。