arXiv:2505.19187cs.CLcs.AI2025-05NeurIPS被引 15

通过精炼推理步骤,让大模型更省力地做出准确判断。

LIMOPro: Reasoning Refinement for Efficient and Effective Test-time Scaling

  • 用困惑度评估每步推理重要性,只删冗余步骤,保留核心解题路径。
  • 在多个难题数据集上,准确率提升0.9%至6.6%,耗 token 减少3%至41%。
  • 适合追求高效推理、低延迟部署的场景,通用性强,适配不同模型和预算。

大型语言模型(LLMs)在测试时通过扩展推理能力展现出强大表现,尤其在使用来自更强大推理模型(LRMs)的链式思维(CoT)数据微调后。然而,这些推理链常包含冗长内容,包括逐步推理(核心解题路径)和功能性元素(验证、备选方案、纠错等),后者显著增加推理计算开销。本文提出基于困惑度的重要度精炼框架PIR,定量评估每一步对答案置信度的影响,系统识别并仅剔除低重要性的功能性步骤,同时保留逐步推理成分,生成优化后的训练数据。在该数据上微调的模型展现出更强的测试时扩展能力:推理链更简洁,准确率提升0.9%至6.6%,令牌消耗减少3%至41%,在AIME、AMC和GPQA Diamond等高难度推理基准上表现优异。该方法在不同模型规模、数据源和令牌预算下均具强泛化性,为需要高效测试时扩展、响应速度与计算效率的推理型大模型部署提供了实用解决方案。

原文摘要 · Abstract (English)

Large language models (LLMs) have demonstrated remarkable reasoning capabilities through test-time scaling approaches, particularly when fine-tuned with chain-of-thought (CoT) data distilled from more powerful large reasoning models (LRMs). However, these reasoning chains often contain verbose elements that mirror human problem-solving, categorized as progressive reasoning (the essential solution development path) and functional elements (verification processes, alternative solution approaches, and error corrections). While progressive reasoning is crucial, the functional elements significantly increase computational demands during test-time inference. We introduce PIR (Perplexity-based Importance Refinement), a principled framework that quantitatively evaluates the importance of each reasoning step based on its impact on answer prediction confidence. PIR systematically identifies and selectively prunes only low-importance functional steps while preserving progressive reasoning components, creating optimized training data that maintains the integrity of the core solution path while reducing verbosity. Models fine-tuned on PIR-optimized data exhibit superior test-time scaling properties, generating more concise reasoning chains while achieving improved accuracy (+0.9\% to +6.6\%) with significantly reduced token usage (-3\% to -41\%) across challenging reasoning benchmarks (AIME, AMC, and GPQA Diamond). Our approach demonstrates strong generalizability across different model sizes, data sources, and token budgets, offering a practical solution for deploying reasoning-capable LLMs in scenarios where efficient test-time scaling, response time, and computational efficiency are valuable constraints.

推理优化大模型高效生成测试时扩展

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。