arXiv:2604.10513cs.AI2026-04

通过语义轨迹分析自动优化AI代理提示,提升任务执行准确性

Agent Mentor: Framing Agent Knowledge through Semantic Trajectory Analysis

  • 基于执行日志分析语义特征,动态修正代理的系统提示
  • 在三种配置下均实现可测量的性能提升,尤其改善模糊指令场景
  • 开源工具链支持未来智能体治理框架的自动化监管

AI代理开发高度依赖自然语言提示来定义任务、知识和目标,这些提示由大语言模型(LLMs)解析并决定代理行为。因此,代理性能易受提示不精准或模糊的影响。发现问题需分析代理代码及执行过程中生成的内部系统提示,体现在执行日志中。本文提出一个集成于Agent Mentor开源库的分析流水线,用于监控并逐步调整另一代理的行为提示。该流水线通过系统性注入纠正指令来改进代理知识。我们阐述其机制,重点在于识别与不良行为相关的语义特征,并据此生成纠正陈述。在三个典型代理配置和基准任务上,通过重复执行验证有效性。实验初步探索了未来智能体治理框架中此类辅导流水线的自动化可能。整体表明,该方法在多种配置下表现出一致且可量化的准确率提升,尤其在规范模糊的场景中效果显著。为确保可复现性,代码已作为Agent Mentor库开源发布。

原文摘要 · Abstract (English)

AI agent development relies heavily on natural language prompting to define agents' tasks, knowledge, and goals. These prompts are interpreted by Large Language Models (LLMs), which govern agent behavior. Consequently, agentic performance is susceptible to variability arising from imprecise or ambiguous prompt formulations. Identifying and correcting such issues requires examining not only the agent's code, but also the internal system prompts generated throughout its execution lifecycle, as reflected in execution logs. In this work, we introduce an analytics pipeline implemented as part of the Agent Mentor open-source library that monitors and incrementally adapts the system prompts defining another agent's behavior. The pipeline improves performance by systematically injecting corrective instructions into the agent's knowledge. We describe its underlying mechanism, with particular emphasis on identifying semantic features associated with undesired behaviors and using them to derive corrective statements. We evaluate the proposed pipeline across three exemplar agent configurations and benchmark tasks using repeated execution runs to assess effectiveness. These experiments provide an initial exploration of automating such a mentoring pipeline within future agentic governance frameworks. Overall, the approach demonstrates consistent and measurable accuracy improvements across diverse configurations, particularly in settings dominated by specification ambiguity. For reproducibility, we released our code as open source under the Agent Mentor library.

智能体提示优化语义分析开源工具

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。