用反向推理提升大模型逻辑能力，不增加计算成本。
Reversal of Thought: Enhancing Large Language Models with Preference-Guided Reverse Reasoning Warm-up
- 通过反向推理与偏好自评，在预热阶段生成任务专属提示。
- 在数学和复杂逻辑任务上准确率超越现有方法，效率更高。
- 适合需要高效精准推理的场景，如教育、智能客服。
大语言模型在推理任务中表现优异，但在数学和复杂逻辑推理方面仍存在局限。现有方法或依赖可追踪的逻辑序列以提升可靠性但增加计算开销，或采用僵化的逻辑模板规则，降低灵活性。本文提出一种即插即用、低成本的推理框架 Reversal of Thought（RoT），用于在批量推理前的预热阶段增强大模型的逻辑推理能力。RoT 采用偏好引导的反向推理预热策略，通过元认知机制整合逻辑符号进行伪代码规划，并利用成对偏好自评估，仅通过示范生成任务特定提示，契合由强化学习人类反馈（RLHF）塑造的大模型认知偏好。通过反向推理，引入认知偏好管理器评估知识边界，聚合已知任务的解题逻辑与未知任务的风格模板，进一步拓展推理能力。多任务实验表明，RoT 在推理准确率与效率上均优于现有基线。
原文摘要 · Abstract (English)
Large language models (LLMs) have shown remarkable performance in reasoning tasks but face limitations in mathematical and complex logical reasoning. Existing methods to improve LLMs' logical capabilities either involve traceable or verifiable logical sequences that generate more reliable responses by constructing logical structures yet increase computational costs, or introduces rigid logic template rules, reducing flexibility. In this paper, we propose Reversal of Thought (RoT), a plug-and-play and cost-effective reasoning framework designed to enhance the logical reasoning abilities of LLMs during the warm-up phase prior to batch inference. RoT utilizes a Preference-Guided Reverse Reasoning warm-up strategy, which integrates logical symbols for pseudocode planning through meta-cognitive mechanisms and pairwise preference self-evaluation to generate task-specific prompts solely through demonstrations, aligning with LLMs' cognitive preferences shaped by RLHF. Through reverse reasoning, we utilize a Cognitive Preference Manager to assess knowledge boundaries and further expand LLMs' reasoning capabilities by aggregating solution logic for known tasks and stylistic templates for unknown tasks. Experiments across various tasks demonstrate that RoT surpasses existing baselines in both reasoning accuracy and efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。