让大模型像苏格拉底一样自省，逐步优化推理过程
SSR: Socratic Self-Refine for Large Language Model Reasoning
- 将回答拆成可验证的子问题与子答案对，逐步评估可信度
- 在五个基准上超越现有方法，准确率提升显著
- 适合需要高可靠推理的场景，如医疗、金融决策
大语言模型虽具强大推理能力，但现有测试时框架多依赖粗粒度自验证与修正，难以应对复杂任务。本文提出苏格拉底式自精炼（SSR）框架，将模型输出分解为可验证的子问题-子答案对，通过受控重解和自一致性检验实现步骤级置信度估计。精准定位不可靠推理步骤并迭代修正，生成更准确、可解释的推理链。在五个推理基准和三种大模型上的实验表明，SSR持续优于当前最优的迭代自精炼基线。此外，SSR提供了一种可解释的黑箱评估方法，用于理解大模型内部推理机制。代码已公开于 https://github.com/SalesforceAIResearch/socratic-self-refine-reasoning。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have demonstrated remarkable reasoning abilities, yet existing test-time frameworks often rely on coarse self-verification and self-correction, limiting their effectiveness on complex tasks. In this paper, we propose Socratic Self-Refine (SSR), a novel framework for fine-grained evaluation and precise refinement of LLM reasoning. Our proposed SSR decomposes model responses into verifiable (sub-question, sub-answer) pairs, enabling step-level confidence estimation through controlled re-solving and self-consistency checks. By pinpointing unreliable steps and iteratively refining them, SSR produces more accurate and interpretable reasoning chains. Empirical results across five reasoning benchmarks and three LLMs show that SSR consistently outperforms state-of-the-art iterative self-refinement baselines. Beyond performance gains, SSR provides a principled black-box approach for evaluating and understanding the internal reasoning processes of LLMs. Code is available at https://github.com/SalesforceAIResearch/socratic-self-refine-reasoning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。