arXiv:2603.03297cs.CLcs.AI2026-03被引 1

让大模型在测试时自我反思纠错,提升难题解题能力

TTSR: Test-Time Self-Evolving via Reflection

  • 模型交替扮演学生和教师角色,先试错再反思生成新问题
  • 在数学推理任务上显著提升性能,跨模型架构通用性强
  • 自动记录并修正弱点,适合需要持续优化的智能问答场景

测试时训练(TTT)在推理阶段仅用无标签测试输入微调大语言模型。但现有方法在复杂推理任务中面临两大瓶颈:(1)可学习样本不足,因困难问题生成的伪标签噪声大,导致奖励不稳定;(2)探索效率低,性能提升依赖大量采样,缺乏对失败原因的显式诊断。本文提出TTSR(Test-Time Self-Evolving via Reflection),基于“反思-重构”范式。单一预训练模型在学生与教师角色间交替:学生解题并更新,教师分析失败轨迹并生成更贴近学生能力边界的变体问题。TTSR还维护跨轮次的‘弱点记忆’,将持续存在的缺陷归纳为轻量级‘策略笔记’,附加至后续学生输入中,使诊断知识指导探索,并随问题解决逐步淡化。在多个高难度数学推理基准测试中,实现稳定测试时性能提升,具备强跨骨干网络泛化能力,并可迁移至通用领域推理任务。

原文摘要 · Abstract (English)

Test-time training (TTT) adapts large language models (LLMs) during inference using only unlabeled test inputs. Existing methods, however, face two major bottlenecks on hard reasoning tasks: (1) \emph{lack of learnable samples}, as self-generated pseudo-labels on difficult questions are often noisy and yield unstable rewards; and (2) \emph{inefficient exploration}, as performance gains depend on repeatedly sampling many rollouts without explicit diagnosis of why previous attempts fail. We propose \textbf{TTSR} (\textbf{T}est-\textbf{T}ime \textbf{S}elf-\textbf{R}eflection), a self-evolving framework based on a \emph{reflect-then-synthesize} paradigm. A single pretrained model alternates between a \textit{Student} role and a \textit{Teacher} role: the Student solves test questions and updates, while the Teacher analyzes failed trajectories and synthesizes targeted variant questions closer to the Student's capability frontier. TTSR further maintains a cross-iteration \textit{weakness memory} and compiles persistent weaknesses into a lightweight \textit{strategy note} prepended to subsequent Student inputs, so diagnostic knowledge can guide exploration and gradually fade as weaknesses are resolved. Experiments on challenging mathematical reasoning benchmarks show consistent test-time improvements, strong cross-backbone generalization, and transfer to general-domain reasoning tasks.

大模型推理自进化测试时训练

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。