研究大模型路由如何影响用户行为与成本权衡。
Routing, Cascades, and User Choice for LLMs
- 构建用户与提供商的博弈模型，分析任务路由策略。
- 最优路由为静态策略，几乎无需多轮重试或级联。
- 发现双方目标不一致时，会损害用户体验。
为缓解性能与成本之间的权衡，大模型服务商根据任务难度和延迟将用户请求路由至不同模型。本文研究了大模型路由对用户行为的影响。我们建立了一个包含两个模型（标准模型与推理模型）的提供商与用户之间的斯塔克尔伯格博弈：若被路由的模型无法解决任务，用户可选择重新提示或放弃任务。用户的目标是最大化效用减去模型使用延迟，而提供商则希望最小化服务成本。通过完全刻画用户的最优响应并简化提供商问题，我们发现，在几乎所有情况下，最优路由策略均为静态策略，且无需级联机制，该策略仅依赖于模型对用户的预期效用。此外，我们揭示了当用户与提供商对模型的效用与成本排序不一致时，存在目标错配间隙。最后，我们展示了极端错配情形：提供商可能通过降低模型延迟来减少自身成本，从而牺牲用户效用。研究结果为单提供商-单用户场景提供了简洁的阈值规则，并阐明了路由、级联与限流的适用条件。
原文摘要 · Abstract (English)
To mitigate the trade-offs between performance and costs, LLM providers route user tasks to different models based on task difficulty and latency. We study the effect of LLM routing with respect to user behavior. We propose a game between an LLM provider with two models (standard and reasoning) and a user who can re-prompt or abandon tasks if the routed model cannot solve them. The user's goal is to maximize their utility minus the delay from using the model, while the provider minimizes the cost of servicing the user. We solve this Stackelberg game by fully characterizing the user best response and simplifying the provider problem. We observe that in nearly all cases, the optimal routing policy involves a static policy with no cascading that depends on the expected utility of the models to the user. Furthermore, we reveal a misalignment gap between the provider-optimal and user-preferred routes when the user's and provider's rankings of the models with respect to utility and cost differ. Finally, we demonstrate conditions for extreme misalignment where providers are incentivized to throttle the latency of the models to minimize their costs, consequently depressing user utility. The results yield simple threshold rules for single-provider, single-user interactions and clarify when routing, cascading, and throttling help or harm.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。