让大模型在测试时通过反馈自动优化推理，提升复杂任务成功率。
Learning to Reason from Feedback at Test-Time
- 将反馈利用建模为测试时的优化问题，动态调整推理路径。
- 在四个推理数据集上，比现有方法更稳定，支持长文本泛化。
- 适合需要高可靠推理的智能系统，如自主决策或科学计算。
大语言模型在单次尝试中解决复杂任务具有挑战性，通常需要与环境迭代交互并获取反馈才能成功，因此有效利用反馈成为关键课题。现有方法要么在长度泛化上表现不佳，要么依赖简单的重试策略，未充分利用历史信息。本文提出FTTT新范式，将测试时的反馈利用形式化为优化问题，并设计可学习的测试时优化器OpTune，以高效挖掘反馈价值。在两种大模型、四个推理数据集上的实验表明，FTTT与OpTune展现出更优的可扩展性与性能表现。
原文摘要 · Abstract (English)
Solving complex tasks in a single attempt is challenging for large language models (LLMs). Iterative interaction with the environment and feedback is often required to achieve success, making effective feedback utilization a critical topic. Existing approaches either struggle with length generalization or rely on naive retries without leveraging prior information. In this paper, we introduce FTTT, a novel paradigm that formulates feedback utilization as an optimization problem at test time. Additionally, we propose a learnable test-time optimizer, OpTune, to effectively exploit feedback. Experiments on two LLMs across four reasoning datasets demonstrate that FTTT and OpTune achieve superior scalability and performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。