让生成模型按需选型，省钱又不降效果。
PromptWise: Online Learning for Cost-Aware Prompt Assignment in Generative Models
- 用在线学习动态匹配提示与最便宜的可用模型
- 在代码生成等任务中成本降低显著，性能相当
- 适合关注推理成本的AI应用开发者
生成式AI的快速发展为用户提供了多种训练好的模型以应对不同提示。在为特定提示选择模型时，用户不仅应考虑性能，还应权衡服务成本。然而，现有模型选择方法通常侧重性能而忽略成本差异。本文提出PromptWise，一种面向成本感知的在线学习框架，通过估计提示-模型兼容性，选择预期能产生满意输出且成本最低的模型。不同于传统上下文老虎机在每个提示上仅做一次决策，PromptWise采用成本感知的老虎机结构，允许对单个提示进行多轮模型分配，从而降低总服务成本。在代码生成和翻译等任务上的数值实验表明，PromptWise在保持与基线方法相当性能的同时，显著降低使用成本。代码已开源：github.com/yannxiaoyanhu/PromptWise。
原文摘要 · Abstract (English)
The rapid advancement of generative AI has provided users with a wide range of well-trained models to address diverse prompts. When selecting a model for a given prompt, users should weigh not only its performance but also its service cost. However, existing model-selection methods typically emphasize performance while overlooking cost differences. In this paper, we introduce PromptWise, an online learning framework that assigns prompts to generative models in a cost-aware manner. PromptWise estimates prompt-model compatibility to select the least expensive model expected to deliver satisfactory outputs. Unlike standard contextual bandits that make a one-shot decision per prompt, PromptWise employs a cost-aware bandit structure that allows sequential model assignments per prompt to reduce total service cost. Through numerical experiments on tasks such as code generation and translation, we demonstrate that PromptWise can achieve performance comparable to baseline selection methods while incurring substantially lower costs. The code is available at: github.com/yannxiaoyanhu/PromptWise.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。