让大模型学会从错误中吸取经验，跨题目提升解题能力。
ReTreVal: Reasoning Tree with Validation and Cross-Problem Memory for Large Language Models
- 构建可自修正的推理树，失败时回溯并注入错误类型信息。
- 在MATH-500上达到85.8%的准确率，比最强基线高8.6个百分点。
- 无需微调，32B模型媲美更大规模单次通过系统，适合实用部署。
现有推理框架在问题边界丢弃所有失败上下文，导致模型解决第500题时与第1题一样无知。我们提出ReTreVal（带验证的推理树），一种无需训练的框架，通过工具增强的节点精炼、分类失败回溯以及自重构记忆，实现跨问题学习。该框架在不微调任何固定大模型的前提下，使模型能积累并修正策略。在MATH-500上达到85.8% pass@1，较零样本思维链提升8.6个百分点，较最强基线Self-Refine高8.6个百分点；在MMLU-Pro上达54.4%，较Self-Refine高出15.3个百分点，3.4:1的胜率对回归比证实了真正的错误修复而非噪声。此前需梯度更新的能力，现可在32B模型上实现，媲美更大规模单次通过系统。
原文摘要 · Abstract (English)
Every existing inference-time reasoning framework discards all failure context at problem boundaries, leaving a model solving problem 500 no wiser than it was on problem 1. We present ReTreVal (Reasoning Tree with Validation), a training-free framework that closes this gap through adaptive tree exploration with tool-augmented node refinement, typed-failure backtracking that injects categorized error context into the recovered branch, and a self-rewriting memory that accumulates and revises strategy entries across problems, enabling inference-time cross-problem learning on any fixed, unmodified LLM without fine-tuning. ReTreVal achieves 85.8% pass@1 on MATH-500 (+8.6 pp over Zero-Shot CoT, +8.6 pp over the strongest baseline Self-Refine) and 54.4% on MMLU-Pro (+15.3 pp over Self-Refine), with a 3.4:1 win-to-regression ratio confirming genuine error recovery rather than noise. These capabilities, previously requiring gradient updates, allow a 32B model to compete with much larger single-pass systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。