arXiv:2607.18973cs.CLcs.AI2026-07

通过预测用户未来反馈来实现对话技能的可验证自进化。

Verifiable Self-Evolution for Open-Ended Dialogue Skills via Future-Feedback Prediction

论文配图:Verifiable Self-Evolution for Open-Ended Dialogue Skills via Future-Feedback Prediction
图 1 · 摘自论文原文
  • 用未来反馈预测代替当前回复生成,实现可验证的自进化
  • 在私有销售助手数据集上达到75%以上预测准确率
  • 适合需要离线优化且避免真实流量测试的对话系统研发

文本技能为提升冻结语言模型代理提供了一种轻量方式,但其自进化通常依赖稳定的验证信号。在数学或代码任务中,答案更改后可直接验证结果;但在开放域对话中,改变回复会引发用户反应变化,导致历史反馈无法评估反事实回复。本文提出未来反馈技能进化方法,将自进化从生成当前回答转向预测当前回答是否会导致正向或负向用户后续反馈。该预测任务可在固定日志样本上验证,支持基于验证的文本优化。演化出的反馈技能捕获了可解释的回复质量标准,可作为回答技能的诊断与优化目标。在某私有、隐私保护的销售助理数据集上,经严格质量过滤和平衡处理后,预测准确率超过75%。核心贡献在于将动态对话反馈转化为静态离线学习目标,实现无需部署到真实流量的可复现技能进化。本文讨论了观测验证与反事实有效性之间的边界,强调该方法为离线优化阶段,而非最终人工或在线评估的替代。

原文摘要 · Abstract (English)

Textual skills provide a lightweight way to improve frozen language-model agents, but their self-evolution normally requires a stable validation signal. Such signals are natural in mathematics or code, where an answer can be checked after it changes, yet are problematic in open-ended dialogue: changing the assistant response also changes the user's next reaction, so a logged reaction cannot directly evaluate a counterfactual response. We propose future-feedback skill evolution, which first redirects self-evolution from prescribing the current answer to predicting whether the observed answer will lead to a positive or negative subsequent user signal. This prediction task is verifiable on fixed logged tuples and therefore supports validation-gated textual optimization. The evolved feedback skill captures interpretable criteria for response quality and can subsequently serve as a diagnostic and optimization target for answer skills. On a proprietary, privacy-preserving sales-assistant dataset, careful quality filtering and a balanced resolved/unresolved split yield more than 75% prediction accuracy. Beyond this result, the central contribution is a formulation that converts otherwise moving conversational feedback into a fixed offline learning target, enabling reproducible skill evolution without placing every candidate skill in live traffic. We discuss the boundary between observational verification and counterfactual validity, and position the method as an offline optimization stage rather than a replacement for final human or online evaluation.

对话系统自进化反馈预测离线优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。