针对大模型配置评估预算有限的问题,提出兼顾成本与多目标的智能选择方法。
Cost-Aware Multi-Objective Bandits: Theory and Application to Budgeted LLM Configuration Evaluation
- 将配置评估建模为带成本的多目标强化学习问题,用超体积效率优化决策。
- 理论证明在预算内可实现对最优配置的高效识别,误差随预算指数下降。
- 适用于资源受限场景下的大模型调优,尤其适合需平衡性能与成本的研究者。
大语言模型(LLM)配置评估面临预算有限、成本差异和多目标冲突的挑战。本文将该问题形式化为一种考虑成本的多目标贝叶斯问题,其中每个配置评估具有依赖于配置的成本,并产生带有噪声的向量结果。在此框架下,研究了两个核心问题:在线配置选择与帕累托配置识别。针对在线选择,提出基于超体积的UCB算法,以优化乐观的单位成本超体积指标。建立了预算遗憾上界为$O\bigl(\sum_{i\ne i^\star}\frac{\log B}{Δ_i}\bigr)$,其中$B$为评估预算,$i^\star$为超体积效率最优配置,$Δ_i$为其效率差距。该界保持了经典单目标预算带子问题中对数预算依赖特性。对于固定预算下的帕累托识别,设计了一种成本感知的经验间隙消除算法,并证明其错误概率为$O\bigl(\exp(-\frac{B}{H_{μ,c}})\bigr)$,其中$H_{μ,c}$为依赖于配置成本与帕累托分类差距的成本感知复杂度。该错误概率随预算呈指数衰减,当所有配置成本相同时,恢复标准帕累托集识别保证。在多个大模型配置评估任务上的实验表明,所提框架可在有限预算下实现高效的在线决策与精确的代价感知帕累托识别。
原文摘要 · Abstract (English)
Large language model (LLM) configuration evaluation is challenging due to limited evaluation budgets, varying costs, and multiple competing objectives. In this paper, we formulate LLM configuration evaluation as a cost-aware multi-objective bandit problem, where each configuration evaluation incurs a configuration-dependent cost and yields a noisy vector-valued outcome. Under this framework, we study two fundamental problems: online configuration selection and Pareto configuration identification. For online configuration selection, we propose a hypervolume-based UCB algorithm that optimizes an optimistic hypervolume-per-cost index. We establish a budgeted regret bound of order $O\bigl(\sum_{i\ne i^\star}\frac{\log B}{Δ_i}\bigr)$, where $B$ is the evaluation budget, $i^\star$ is the optimal configuration in terms of hypervolume efficiency, and $Δ_i$ is the corresponding efficiency gap of configuration $i$. This bound retains the logarithmic budget dependence of classical single-objective budgeted bandits. For fixed-budget Pareto identification, we develop a cost-aware empirical gap elimination algorithm and prove that its error probability is of order $O\bigl(\exp(-\frac{B}{H_{μ,c}})\bigr)$, where $H_{μ,c}$ is a cost-aware Pareto identification complexity depending on configuration costs and Pareto classification gaps. This error probability decays exponentially with the evaluation budget and recovers the standard Pareto set identification guarantee when all configuration costs are identical. Experiments on LLM configuration evaluation tasks demonstrate that the proposed framework enables efficient online decision-making and accurate cost-aware Pareto identification under limited budgets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。