arXiv:2411.01855cs.CL2024-11NeurIPS被引 92

让语言模型学会跳步推理,提升效率且不丢准确率

Can Language Models Learn to Skip Steps?

  • 通过迭代优化引导模型生成更短的推理路径
  • 在扩展数据集上微调后,推理效率提升且保持高准确率
  • 首次实现模型类人跳步,适合研究高效AI推理的学者

语言模型虽具备类人推理能力,但尚未具备主动跳过冗余步骤以提升效率的动机。本文提出一个可控框架,通过迭代精炼使模型生成更短且准确的推理路径。实验证明,在包含完整与跳步推理序列的扩展数据集上微调后,模型不仅推理效率显著提升,且准确率不降,还展现出更强的跨领域泛化能力。该工作首次探索了类人跳步推理机制,为提升AI推理效率提供了新思路。

原文摘要 · Abstract (English)

Trained on vast corpora of human language, language models demonstrate emergent human-like reasoning abilities. Yet they are still far from true intelligence, which opens up intriguing opportunities to explore the parallels of humans and model behaviors. In this work, we study the ability to skip steps in reasoning - a hallmark of human expertise developed through practice. Unlike humans, who may skip steps to enhance efficiency or to reduce cognitive load, models do not inherently possess such motivations to minimize reasoning steps. To address this, we introduce a controlled framework that stimulates step-skipping behavior by iteratively refining models to generate shorter and accurate reasoning paths. Empirical results indicate that models can develop the step skipping ability under our guidance. Moreover, after fine-tuning on expanded datasets that include both complete and skipped reasoning sequences, the models can not only resolve tasks with increased efficiency without sacrificing accuracy, but also exhibit comparable and even enhanced generalization capabilities in out-of-domain scenarios. Our work presents the first exploration into human-like step-skipping ability and provides fresh perspectives on how such cognitive abilities can benefit AI models.

推理优化语言模型跳步推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。