arXiv:2607.08665cs.LG2026-07

在预算有限时,动态选择重采样或切换模型,提升大模型推理质量。

Resample or Reroute? Budget-Aware Test-Time Model Selection for Large Language Models

  • 根据预算分配,在重采样和换模型间动态决策。
  • 在四个基准上均优于单一路由等基线方法,尤其在异构模型中增益显著。
  • 适合资源受限但追求高质量输出的LLM部署场景。

在大型语言模型中,路由策略需权衡响应质量与服务成本。尽管测试时重采样可恢复单实例最优选择的潜力,但这一优势依赖于理想化的正确性标签和无约束预算,现实系统无法满足。本文首次将重采样与路由视为同一查询预算下的竞争选择,提出预算感知的测试时模型选择:给定每查询预算和不完美的验证器,优化每一单位预算在重采样与路由间的分配以最大化预期正确率。提出一种基于边际正确率-成本比的在线重采样或路由(RoR)策略,其设计基于选择与采样之间可恢复性的不对称性。在包含11个开源模型的池上,对四个难度不同的基准进行重生成多轮采样正确性张量的回放实验表明,所提RoR策略在成本-质量帕累托前沿上优于单一路由、单次提交路由器、预算感知Best-of-K、级联和随机分配等基线;在最异构的基准上收益最大。消融实验显示收益受验证器质量限制,随其下降而缩小;在提供商价格向量和无标签一致性验证器下的鲁棒性测试进一步明确了结论的有效范围。

原文摘要 · Abstract (English)

Routing among large language models (LLMs) trades response quality against serving cost, motivated by the reported gap between deployed routers and a per-instance oracle. Recent analysis shows that test-time resampling can recover per-instance selection headroom that no single-commit router captures; however, that guarantee holds only under an idealized oracle equipped with correctness labels and an unconstrained budget, neither of which a deployed system has. To the best of our knowledge, no previous work treats resampling the committed model and rerouting to an alternative model as competing uses of a single per-query cost budget. Therefore, this work formulates budget-aware test-time model selection: given a per-query budget and an imperfect verifier, allocate each unit of budget between resampling and rerouting so that expected correctness is maximized. An online resample-or-reroute (RoR) allocation policy driven by estimated marginal correctness per unit cost is proposed, and its behavior is grounded in the recoverability asymmetry between selection and sampling. Replay experiments on newly regenerated multi-draw correctness tensors from an eleven-model open-weight pool over four benchmarks of differing difficulty show that the proposed RoR policy attains a favorable cost-quality Pareto front relative to single-route, one-commit-router, budget-aware best-of-K, cascade, and random-allocation baselines for the tested pools, with the largest gains on the most heterogeneous benchmark; an ablation further shows the gains are verifier-gated, shrinking as verifier quality degrades, and robustness replays under a provider price vector and a label-free agreement verifier delineate where the conclusions carry over.

大模型推理预算优化模型选择

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。