让大模型学会按需计算，节省资源同时提升推理效果。
ROI-Reasoning: Rational Optimization for Inference via Pre-Computation Meta-Cognition
- 先预估任务难度和收益，决定是否解题
- 在严格字数限制下，得分更高且浪费更少
- 适合需要高效推理的实时应用
大型语言模型（LLMs）在充足计算下可实现强大推理能力，但缺乏对任务所需计算量的内在认知。本文研究在严格全局词元约束下多个任务的预算化推理，将其形式化为有序随机多选背包问题（OS-MCKP）。该视角凸显了元认知需求——预判任务难度、估算投入产出比（ROI）、战略分配计算资源。我们提出ROI-Reasoning，一种两阶段框架，赋予模型内在的预算感知理性。第一阶段通过元认知微调，使模型在生成前预测推理成本与预期效用，实现显式的解或跳决策；第二阶段通过理性感知强化学习，在硬性词元预算下优化序列决策，使模型学习长程分配策略。在多个预算化数学推理基准上，ROI-Reasoning持续提升总分，同时大幅降低紧约束下的遗憾值。
原文摘要 · Abstract (English)
Large language models (LLMs) can achieve strong reasoning performance with sufficient computation, but they do not inherently know how much computation a task requires. We study budgeted inference-time reasoning for multiple tasks under a strict global token constraint and formalize it as a Ordered Stochastic Multiple-Choice Knapsack Problem(OS-MCKP). This perspective highlights a meta-cognitive requirement -- anticipating task difficulty, estimating return over investment (ROI), and allocating computation strategically. We propose ROI-Reasoning, a two-stage framework that endows LLMs with intrinsic, budget-aware rationality. In the first stage, Meta-Cognitive Fine-Tuning teaches models to predict reasoning cost and expected utility before generation, enabling explicit solve-or-skip decisions. Next, Rationality-Aware Reinforcement Learning optimizes sequential decision making under a hard token budget, allowing models to learn long-horizon allocation strategies. Across budgeted mathematical reasoning benchmarks, ROI-Reasoning consistently improves overall score while substantially reducing regret under tight computation budgets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。