利用用户修正过程提炼真实交互数据,持续提升AI代理性能。
Echo: Learning from Experience Data via User-Driven Refinement

- 通过用户修正序列提取高质量训练信号
- 代码生成任务中采纳率从25.7%提升至35.7%
- 适合需要持续优化的部署型AI系统
静态人类数据存在成本高、知识受限的问题。持续学习来自智能体与环境交互的‘经验数据’,有望突破这些瓶颈。如今大规模部署的AI代理带来了低成本的海量真实交互流。但原始日志噪声大、信息密度低,难以直接用于训练。我们提出Echo框架,将原始经验转化为可学习知识,实现环境反馈的闭环优化。在实际代码生成场景中,用户修正成为核心反馈来源:用户为结果负责,会严格将错误提案转化为有效解。这种用户驱动的修正过程天然将粗糙尝试提炼为高质量训练信号。Echo系统性地捕获这些信号,持续对齐代理与真实需求。大规模验证表明,该方法成功打破静态性能瓶颈,使采纳率从25.7%提升至35.7%。
原文摘要 · Abstract (English)
Static "human data" faces inherent limitations: it is expensive to scale and bounded by the knowledge of its creators. Continuous learning from "experience data" - interactions between agents and their environments - promises to transcend these barriers. Today, the widespread deployment of AI agents grants us low-cost access to massive streams of such real-world experience. However, raw interaction logs are inherently noisy, filled with trial-and-error and low information density, rendering them inefficient for direct model training. We introduce Echo, a generalized framework designed to operationalize the transition from raw experience to learnable knowledge, effectively "echoing" environmental feedback back into the training loop for model optimization. In today's agent ecosystem, user refinement serves as a primary source of such feedback: driven by responsibility for the outcome, users rigorously transform flawed agent proposals into verified solutions. These user-driven refinement sequences inherently distill agents' crude attempts into high-quality training signals. Echo systematically harvests these signals to continuously align the agent with real-world needs. Large-scale validation in a production code completion environment confirms that Echo effectively harnesses this pipeline, breaking the static performance ceiling by increasing the acceptance rate from 25.7% to 35.7%.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。