让大模型根据输入难度自动调节计算量，省一半资源也不降低效果。
Learning How Hard to Think: Input-Adaptive Allocation of LM Computation
- 根据输入难易动态分配计算资源，难问题多算，简单问题少算。
- 在编程、数学和对话任务上，计算量减少50%且质量不变，或同等预算下质量提升10%。
- 适用于需要高效推理的场景，如实时对话系统或大规模部署。
计算密集型解码过程——包括搜索、重排序和自省——可在代码生成、数值推理和对话等任务中提升语言模型输出质量。现有方法对所有输入使用相同的解码流程，但并非所有输入都需要同等计算量。能否根据输入难度自适应分配计算？本文提出一种方法：预测给定输入与计算预算下的奖励分布，并将额外计算资源分配给预期收益最大的输入。该方法应用于两种解码流程：一是自适应的best-of-k策略，动态决定生成样本数以供重排序；二是路由机制，根据查询动态选择高精度但昂贵的解码方式或低成本但能力较弱的方式。在涵盖编程、数学和对话的一系列任务中，我们证明了可学习准确的计算分配策略，在不降低响应质量的前提下，计算量最多减少50%，或在固定计算预算下，质量最高提升10%。
原文摘要 · Abstract (English)
Computationally intensive decoding procedures--including search, reranking, and self-critique--can improve the quality of language model (LM) outputs in problems spanning code generation, numerical reasoning, and dialog. Existing work typically applies the same decoding procedure for every input to an LM. But not all inputs require the same amount of computation to process. Can we allocate decoding computation adaptively, using more resources to answer questions whose answers will be harder to compute? We present an approach that predicts the distribution of rewards given an input and computation budget, then allocates additional computation to inputs for which it is predicted to be most useful. We apply this approach in two decoding procedures: first, an adaptive best-of-k procedure that dynamically selects the number of samples to generate as input to a reranker; second, a routing procedure that dynamically responds to a query using a decoding procedure that is expensive but accurate, or one that is cheaper but less capable. Across a suite of programming, mathematics, and dialog tasks, we show that accurate computation-allocation procedures can be learned, and reduce computation by up to 50% at no cost to response quality, or improve quality by up to 10% at a fixed computational budget.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。