通过多维度变化测试模型推理一致性，提升小模型数学能力。
Multidimensional Consistency Improves Reasoning in Language Models
- 设计多维度输入扰动（顺序、表述、语言）检验推理一致性。
- 在GSM8K和MGSM上，一致性聚合使小模型性能显著提升。
- 适合关注模型鲁棒性与小模型优化的研究者参考。
尽管大语言模型在解决复杂推理任务方面表现优异，但其对输入变化高度敏感，可能导致不同解题路径与答案。因此，答案在输入变化下的稳定性可作为模型信心的标志。本文提出多维推理一致性框架，聚焦数学问题，系统性地诱导三类输入变化：提示中示例顺序、题目表述方式及使用语言。在多个开源前沿大模型上进行广泛实验表明，推理一致性因变化维度而异；通过聚合多维度一致性，该框架在单语数据集GSM8K和多语数据集MGSM上均显著提升数学推理性能，尤其对小型模型效果更明显。
原文摘要 · Abstract (English)
While Large language models (LLMs) have proved able to address some complex reasoning tasks, we also know that they are highly sensitive to input variation, which can lead to different solution paths and final answers. Answer consistency across input variations can thus be taken as a sign of stronger confidence. Leveraging this insight, we introduce a framework, {\em Multidimensional Reasoning Consistency} where, focusing on math problems, models are systematically pushed to diversify solution paths towards a final answer, thereby testing them for answer consistency across multiple input variations. We induce variations in (i) order of shots in prompt, (ii) problem phrasing, and (iii) languages used. Extensive experiments on a large range of open-source state-of-the-art LLMs of various sizes show that reasoning consistency differs by variation dimension, and that by aggregating consistency across dimensions, our framework consistently enhances mathematical reasoning performance on both monolingual dataset GSM8K and multilingual dataset MGSM, especially for smaller models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。