arXiv:2606.07392cs.AIcs.LG2026-06被引 1

动态选择LLM接口,按输出成本自适应调用,提升响应效率。

Online Pandora's Box for Contextual LLM Cascading

  • 基于上下文构建预留指数,决定何时停止调用接口
  • 理论证明累计后悔值为√T量级,随周期数增长缓慢
  • 适合需要高效调用多个LLM接口的实时系统场景

受大语言模型级联启发,我们提出一种在线上下文式潘多拉魔盒模型,用于自适应查询和选择LLM API。每个周期中,决策者先观察请求上下文,再经历两阶段决策:查询阶段逐个调用API,每次获得生成输出及对应(输出相关)成本;选择阶段从所有生成输出中选一个部署,并仅观测该输出的下游奖励。此反馈机制不同于经典模型中直接揭示盒子奖励的情形。我们不估计各API的完整输出与成本分布,而是直接建模预留指数,并采用参数化结构的上下文预留指数函数。所提策略结合广义矩法(GMM)估计与UCB风格置信区间,同时处理预留指数和共享输出级奖励评估器。在常规条件下,证明该策略在T期内的累计后悔值为维度相关的˜O(√T)。

原文摘要 · Abstract (English)

Motivated by Large Language Model (LLM) cascading, we propose an online contextual Pandora's Box model for adaptively querying and selecting LLM APIs. In each period, a decision-maker observes a request context and faces a two-phase decision problem. In the query phase, the decision-maker sequentially queries APIs, where each query reveals a generated output and the decision-maker incurs an (output-dependent) cost. In the selection phase, the decision-maker selects one of the generated outputs to deploy and observes only the downstream reward of the deployed output. This output-mediated feedback structure differs from classical online contextual Pandora's Box models, in which opening a box directly reveals its reward. Rather than estimating the full conditional output and cost distributions of each API, we directly model the reservation index and develop a learning approach for the query phase. Specifically, we impose a parametric structure on the contextual reservation index functions induced by the classical Weitzman's policy. Our policy combines generalized method of moments (GMM) type estimation of these reservation indices with UCB-style confidence bounds for both these indices and the shared output-level reward evaluator. Under regularity conditions, we prove that the resulting policy achieves dimension-dependent $\widetilde O(\sqrt T)$ cumulative regret over a horizon of $T$ periods.

LLM级联在线学习决策优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。