通过分析推理过程提升大模型逻辑判断力
Semantic Self-Consistency: Enhancing Language Model Reasoning via Semantic Weighting
- 在多数投票前分析多个推理路径的语义信息
- 复杂推理任务准确率显著提升
- 适合需要可靠逻辑推演的应用场景
尽管大语言模型在众多任务上表现优异，但在推理任务上仍存在不足。随着模型在现实任务中广泛应用，提升其推理能力至关重要。Wang等提出的自一致性框架表明，对多个推理路径进行采样并取多数投票可有效提升模型性能。现有方法仅聚合最终答案，未利用推理过程中的语义信息。本文提出语义自一致性，通过分析推理路径与最终决策，在多数投票前融合语义信息，不仅提升了推理路径的可靠性，还在复杂推理任务上实现了更稳健的表现。
原文摘要 · Abstract (English)
While large language models (LLMs) have rapidly improved their performance on a broad number of tasks, they still often fall short on reasoning tasks. As LLMs become more integrated in diverse real-world tasks, advancing their reasoning capabilities is crucial to their effectiveness in nuanced, complex problems. Wang et al.'s self-consistency framework reveals that sampling multiple rationales before taking a majority vote reliably improves model performance across various closed-answer reasoning tasks. Standard methods based on this framework aggregate the final decisions of these rationales but fail to utilize the semantic information detailed in the step-by-step reasoning paths. Our work introduces semantic self-consistency, enhancing this approach by incorporating and analyzing both the reasoning paths of these rationales in addition to their final decisions before taking a majority vote. These methods not only improve the reliability of reasoning paths but also cause more robust performance on complex reasoning tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。