将推理步骤建模为连续概率流,提升大模型推理效率与准确性
Efficient Paths and Dense Rewards: Probabilistic Flow Reasoning for Large Language Models
- 把思维链看作连续概率流,量化每一步对答案的贡献
- 推理速度提升30%,在多个基准测试中表现优于现有方法
- 适合需要高效推理的场景,如实时问答或资源受限部署
高质量思维链已展现出激发大语言模型推理能力的巨大潜力。然而,当前范式通常将推理过程视为不可分割的序列,缺乏内在机制来量化每一步的信息增益。这种粒度缺失导致两个问题:缺乏明确引导的冗余探索造成推理低效,以及稀疏结果监督或依赖昂贵外部验证器带来的优化困难。本文提出CoT-Flow框架,将离散推理步骤重新构想为连续概率流,量化每一步对真实答案的贡献。基于此,CoT-Flow实现了两种互补方法:流引导解码,采用贪婪流策略提取信息高效的推理路径;基于流的强化学习,构建无需验证器的密集奖励函数。在多个挑战性基准上的实验表明,CoT-Flow在推理效率与推理性能之间取得了更优平衡。
原文摘要 · Abstract (English)
High-quality chain-of-thought has demonstrated strong potential for unlocking the reasoning capabilities of large language models. However, current paradigms typically treat the reasoning process as an indivisible sequence, lacking an intrinsic mechanism to quantify step-wise information gain. This granularity gap manifests in two limitations: inference inefficiency from redundant exploration without explicit guidance, and optimization difficulty due to sparse outcome supervision or costly external verifiers. In this work, we propose CoT-Flow, a framework that reconceptualizes discrete reasoning steps as a continuous probabilistic flow, quantifying the contribution of each step toward the ground-truth answer. Built on this formulation, CoT-Flow enables two complementary methodologies: flow-guided decoding, which employs a greedy flow-based decoding strategy to extract information-efficient reasoning paths, and flow-based reinforcement learning, which constructs a verifier-free dense reward function. Experiments on challenging benchmarks demonstrate that CoT-Flow achieves a superior balance between inference efficiency and reasoning performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。