arXiv:2605.11019cs.LGcs.AI2026-05

通过变分后验引导提升大模型推理效率,解决过思考导致的低效问题。

Efficient LLM Reasoning via Variational Posterior Guidance with Efficiency Awareness

论文配图:Efficient LLM Reasoning via Variational Posterior Guidance with Efficiency Awareness
图 1 · 摘自论文原文
  • 将高效推理建模为变分推断问题,引入效率感知的下界理论
  • 双流架构过滤伪高效路径,通过变分蒸馏将后验模式迁移至先验策略
  • 在1.5B和7B模型上分别提升综合效率8.73%和12.37%,优于最强基线

尽管大语言模型依赖思维链进行复杂推理,但过思考现象严重降低推理效率。现有强化学习方法通过设计复杂的奖励函数压缩思维链,导致高质量样本在探索空间中极度稀疏,造成先验策略的采样瓶颈。受认知科学启发,我们从理论上证明:以参考答案为导向的后验分布相比先验分布具有更高的期望效用,因而能突破高质量样本的采样瓶颈。然而后验分布在推理时不可用。为此,我们将高效推理形式化为变分推断问题,提出效率感知的证据下界作为理论基础。基于此,我们构建VPG-EA框架:采用参数共享的双流架构,同时实例化后验分布与先验策略;通过跨视图评估过滤伪高效路径后,单向将后验的高效模式通过变分蒸馏传递给先验策略。在DeepSeek-R1-Distill-Qwen-1.5B和7B规模上的实验表明,VPG-EA分别相较最强基线提升综合效率指标ε³达8.73%和12.37%。

原文摘要 · Abstract (English)

Although large language models rely on chain-of-thought for complex reasoning, the overthinking phenomenon severely degrades inference efficiency. Existing reinforcement learning methods compress reasoning chains by designing elaborate reward functions, which renders high-quality samples extremely sparse in the exploration space and creates a sampling bottleneck for the prior policy. Inspired by cognitive science, we theoretically prove that a posterior distribution guided by reference answers achieves higher expected utility than the prior distribution, thus capable of breaking through the sampling bottleneck of high-quality samples. However, the posterior distribution is unavailable during inference. To this end, we formalize efficient reasoning as a variational inference problem and introduce an efficiency-aware evidence lower bound as the theoretical foundation. Based on this, we propose the VPG-EA framework. It adopts a parameter-shared dual-stream architecture to instantiate both the posterior distribution and the prior policy; after filtering out pseudo-efficient paths via cross-view evaluation, it unidirectionally transfers the posterior's efficient patterns to the prior policy through variational distillation. Experiments on DeepSeek-R1-Distill-Qwen-1.5B and 7B scales demonstrate that VPG-EA improves the comprehensive efficiency metric epsilon cubed by 8.73% and 12.37% over the strongest baselines on each model size, respectively.

大模型推理思维链优化变分推断效率提升

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。