用智能提示+高效模型,只算关键步骤就大幅提速推理
Efficient Thought Space Exploration Through Strategic Intervention
- 用大模型在关键点给小模型提建议,减少无效计算
- 只计算1/5的词元,准确率却接近顶尖方法
- 适合需要快速推理又怕出错的场景
尽管大语言模型展现出日益增强的推理能力,但现有推理时扩展方法因全量采样导致计算成本过高。通过分析解码轨迹,我们发现多数下一步词元预测与标准答案一致,仅少数关键词元会导致偏差。受此启发,提出提示-实践推理(HPR)框架,包含两个协同组件:一个强大的提示者(hinter)在关键决策点提供概率指引,一个高效的实践者(practitioner)执行主要推理步骤。该框架的核心创新是分布不一致性降低(DIR),一种理论基础扎实的度量,通过量化实践者推理路径与提示者预期分布之间的差异,在树状概率空间中动态识别干预点。通过DIR引导的迭代树更新,HPR对有希望的推理路径进行重加权,同时降低低概率分支的优先级。在算术和常识推理基准上的实验表明,HPR在效率-精度权衡上达到当前最佳:性能与自洽性和MCTS基线相当,但仅需1/5的词元解码;在最多5.1%的绝对准确率提升下,保持或低于同等FLOPs。
原文摘要 · Abstract (English)
While large language models (LLMs) demonstrate emerging reasoning capabilities, current inference-time expansion methods incur prohibitive computational costs by exhaustive sampling. Through analyzing decoding trajectories, we observe that most next-token predictions align well with the golden output, except for a few critical tokens that lead to deviations. Inspired by this phenomenon, we propose a novel Hint-Practice Reasoning (HPR) framework that operationalizes this insight through two synergistic components: 1) a hinter (powerful LLM) that provides probabilistic guidance at critical decision points, and 2) a practitioner (efficient smaller model) that executes major reasoning steps. The framework's core innovation lies in Distributional Inconsistency Reduction (DIR), a theoretically-grounded metric that dynamically identifies intervention points by quantifying the divergence between practitioner's reasoning trajectory and hinter's expected distribution in a tree-structured probabilistic space. Through iterative tree updates guided by DIR, HPR reweights promising reasoning paths while deprioritizing low-probability branches. Experiments across arithmetic and commonsense reasoning benchmarks demonstrate HPR's state-of-the-art efficiency-accuracy tradeoffs: it achieves comparable performance to self-consistency and MCTS baselines while decoding only 1/5 tokens, and outperforms existing methods by at most 5.1% absolute accuracy while maintaining similar or lower FLOPs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。