arXiv:2506.22376cs.LGcs.AI2025-06AAAI被引 3

提出可动态调节采样数的推理加速方法,提升大模型推理效率。

OptScale: Probabilistic Optimality for Inference-time Scaling

  • 基于独立同分布假设,构建推理时缩放的理论框架。
  • 在多个数学推理数据集上,采样量减少50%仍保持最优性能。
  • 适合追求高效推理的工业级大模型部署场景。

推理时缩放已成为提升大语言模型(LLMs)推理性能的强大技术。然而,现有方法通常依赖启发式并行采样策略,缺乏理论基础。为此,本文提出一个概率框架,在并行样本独立同分布(i.i.d.)且Best-of-$N$选择策略的概率分布可估计的前提下,形式化推理时缩放的最优性。该框架推导出达到目标性能水平所需的最小样本数下界,为计算高效缩放提供了首个理论指导。基于此,我们开发了 extsc{OptScale}——一种动态确定最优采样响应数量的实用算法。 extsc{OptScale}采用语言模型预测器估计概率先验参数,从而决策满足预设性能阈值与置信水平的最少采样数。在代表性推理基准(包括MATH-500、GSM8K、AIME和AMC)上的大量实验表明, extsc{OptScale}显著降低采样开销,同时性能优于或媲美当前最优水平。本工作为有原则的推理时缩放提供了理论基础与实践方案,填补了复杂推理任务中高效部署大模型的关键空白。

原文摘要 · Abstract (English)

Inference-time scaling has emerged as a powerful technique for enhancing the reasoning performance of Large Language Models (LLMs). However, existing approaches often rely on heuristic strategies for parallel sampling, lacking a principled foundation. To address this gap, we propose a probabilistic framework that formalizes the optimality of inference-time scaling under the assumption that parallel samples are independently and identically distributed (i.i.d.), and where the Best-of-$N$ selection strategy follows a probability distribution that can be estimated. Within this framework, we derive a theoretical lower bound on the required number of samples to achieve a target performance level, providing the first principled guidance for compute-efficient scaling. Leveraging this insight, we develop \textsc{OptScale}, a practical algorithm that dynamically determines the optimal number of sampled responses. \textsc{OptScale} employs a language model-based predictor to estimate probabilistic prior parameters, enabling the decision of the minimal number of samples needed that satisfy predefined performance thresholds and confidence levels. Extensive experiments on representative reasoning benchmarks (including MATH-500, GSM8K, AIME, and AMC) demonstrate that \textsc{OptScale} significantly reduces sampling overhead while remaining better or on par with state-of-the-art reasoning performance. Our work offers both a theoretical foundation and a practical solution for principled inference-time scaling, addressing a critical gap in the efficient deployment of LLMs for complex reasoning.

推理优化概率建模大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。