研究自然语言反馈如何真正提升模型性能,发现多数改进源于重试而非有效反馈。
What Drives Interactive Improvement from Feedback?

- 构建师生对照实验,分离反馈与重复尝试的影响
- 外部教师反馈带来显著提升,自生成反馈效果有限
- 模型能否执行反馈比反馈本身更重要,适合评估交互式AI
我们研究在多轮语言智能体设置中,自然语言反馈是否带来超越单纯重复尝试的改进。在Omni-MATH、Codeforces、BBEH Linguini和ARC-AGI1数据集上,对十三个开源模型进行师生角色切换实验,比较外部反馈、自我反馈与无引导自精炼。结果表明,多轮改进常非反馈所致:自生成反馈带来的增益几乎等同于无引导自精炼;而表现最优的外部教师则产生显著的反馈特异性提升,说明有效反馈需提供超出泛化重试的指导。密集师生交互矩阵显示,交互增益更多取决于学生使用反馈的能力,而非教师身份,但教师选择对固定学生仍关键。研究建议将反馈型智能体与重复尝试基线对比,并强调执行反馈能力是交互改进的核心瓶颈。代码框架已开源。
原文摘要 · Abstract (English)
We study when natural-language feedback produces improvement beyond the gains obtainable from repeated attempts alone. In multi-turn language agent setting, higher final accuracy can reflect useful feedback, but it can also arise from resampling, format correction, or additional test-time computation. To separate these effects, we introduce a controlled student-teacher protocol across Omni-MATH, Codeforces, BBEH Linguini, and ARC-AGI1, evaluating thirteen open-weight models in both student and teacher roles. We compare external feedback, self-feedback, and unguided self-refinement, while varying interaction history, task difficulty, and teacher access to privileged task information. Across settings, we find that multi-turn improvement is often not evidence of feedback use: self-generated feedback adds little beyond unguided self-refinement, whereas the strongest external teachers produce substantially larger feedback-specific gains, suggesting that useful feedback must provide guidance beyond generic retry. Dense student-teacher interaction matrices further show that interactive gains are driven more by the student's ability to use feedback than by the teacher's identity, although teacher choice remains important for a fixed student. These results suggest that feedback-based agents should be evaluated against repeated-attempt baselines, and that ability to act on feedback, not merely feedback availability, is a central bottleneck for interactive improvement. We release our controlled student-teacher evaluation framework at https://j-lojek.github.io/feedback-generation-is-a-bottleneck/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。