根据输入提示动态选最优生成模型,降低无效调用成本。
PAK-UCB Contextual Bandit: An Online Learning Approach to Prompt-Aware Selection of Generative Models and LLMs
- 基于上下文强化学习,实时判断不同提示对应最佳生成模型。
- 在图文生成任务中,相比固定选择,显著提升生成质量与效率。
- 适合需要高效调度多模型的AI应用开发人员使用。
从多个基于提示的生成模型(包括大语言模型和提示引导的图像/视频生成模型)中选择最佳生成方案,传统方法通常依据平均评分挑选模型。然而,该方法忽略了不同模型对不同类型文本提示的表现差异。本文提出一种在线学习框架,用于识别针对不同输入提示表现最优的生成模型。所提出的PAK-UCB算法适用于具有共享上下文变量的上下文老虎机问题,利用生成数据更新基于核函数的预测函数,以预估各模型在未见提示下的得分。此外,通过引入随机傅里叶特征(RFF)加速在线学习过程。真实与模拟的文本到图像、图像到文本生成模型实验表明,RFF-UCB能有效识别不同样本类型下的最优生成模型。代码已开源:github.com/yannxiaoyanhu/dgm-online-select。
原文摘要 · Abstract (English)
Selecting a sample generation scheme from multiple prompt-based generative models, including large language models (LLMs) and prompt-guided image and video generation models, is typically addressed by choosing the model that maximizes an averaged evaluation score. However, this score-based selection overlooks the possibility that different models achieve the best generation performance for different types of text prompts. An online identification of the best generation model for various input prompts can reduce the costs associated with querying sub-optimal models. In this work, we explore the possibility of varying rankings of text-based generative models for different text prompts and propose an online learning framework to predict the best data generation model for a given input prompt. The proposed PAK-UCB algorithm addresses a contextual bandit (CB) setting with shared context variables across the arms, utilizing the generated data to update kernel-based functions that predict the score of each model available for unseen text prompts. Additionally, we leverage random Fourier features (RFF) to accelerate the online learning process of PAK-UCB. Our numerical experiments on real and simulated text-to-image and image-to-text generative models show that RFF-UCB performs successfully in identifying the best generation model across different sample types. The code is available at: github.com/yannxiaoyanhu/dgm-online-select.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。