arXiv:2602.00871cs.AIcs.CL2026-02

让大模型先抽象任务,再纠错,小模型也能高效改进答案。

Beyond Output Critique: Self-Correction via Task Distillation

  • 先将任务提炼成结构化模板,再基于模板修正答案。
  • 小模型使用大模型生成的模板,准确率提升23%以上。
  • 无需微调或外部验证,适合资源有限的部署场景。

大型语言模型具备一定的自我修正能力,但多数方法仅在输出层面修补表面错误,难以解决深层推理缺陷。本文提出SELF-THOUGHT框架,在初始回答后增加任务抽象步骤:模型将输入与初始响应提炼为包含关键变量、约束和问题结构的结构化模板。该抽象作为指导,帮助后续修正过程建立更清晰的任务理解,减少错误传播。关键创新在于,大模型生成的抽象模板可跨模型迁移,使原本难以自纠的小模型获得稳定提升。实验表明,该方法在多种推理任务中显著提高大模型与小模型的准确性、鲁棒性和泛化能力,提供了一条无需重训或依赖外部验证的可扩展自纠正路径。

原文摘要 · Abstract (English)

Large language models (LLMs) have shown promising self-correction abilities, where iterative refinement improves the quality of generated responses. However, most existing approaches operate at the level of output critique, patching surface errors while often failing to correct deeper reasoning flaws. We propose SELF-THOUGHT, a framework that introduces an intermediate step of task abstraction before solution refinement. Given an input and an initial response, the model first distills the task into a structured template that captures key variables, constraints, and problem structure. This abstraction then guides solution instantiation, grounding subsequent responses in a clearer understanding of the task and reducing error propagation. Crucially, we show that these abstractions can be transferred across models: templates generated by larger models can serve as structured guides for smaller LLMs, which typically struggle with intrinsic self-correction. By reusing distilled task structures, smaller models achieve more reliable refinements without heavy fine-tuning or reliance on external verifiers. Experiments across diverse reasoning tasks demonstrate that SELF-THOUGHT improves accuracy, robustness, and generalization for both large and small models, offering a scalable path toward more reliable self-correcting language systems.

自纠错任务抽象小模型优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。