在标签预算有限下，用博弈论优化大模型微调，提升效率与准确率。
Optimal Budgeted Adaptation of Large Language Models
- 将微调建模为上下文斯塔克尔伯格博弈，主动规划标签查询策略。
- 在预算约束下实现近似最优的累积误差，理论证明优于传统方法。
- 适合资源受限场景，如低资源语言或高成本标注任务的模型优化。
在大规模语言模型微调中，标注数据的可用性与下游性能之间的权衡仍是核心挑战。本文提出一种有原则的预算感知监督微调框架，将LLM适应建模为上下文斯塔克尔伯格博弈：学习者（领导者）确定评分策略和标签查询机制，自适应环境（跟随者）则响应选择更具挑战性的监督样本。为显式处理标签效率，将有限的标注预算直接纳入目标函数。算法在全反馈环境下运行，在标准线性上下文假设下达到$ ilde{O}(d oot{T})$的遗憾界。进一步引入最大延迟优先（LLF）置信度门控机制，实现预算感知的遗憾界$ ilde{O}( oot{dB} + c oot{B})$，其中$B=βT$。实验表明该方法在保持高精度的同时显著减少标签请求次数。
原文摘要 · Abstract (English)
The trade-off between labeled data availability and downstream accuracy remains a central challenge in fine-tuning large language models (LLMs). We propose a principled framework for \emph{budget-aware supervised fine-tuning} by casting LLM adaptation as a contextual Stackelberg game. In our formulation, the learner (leader) commits to a scoring policy and a label-querying strategy, while an adaptive environment (follower) selects challenging supervised alternatives in response. To explicitly address label efficiency, we incorporate a finite supervision budget directly into the learning objective. Our algorithm operates in the full-feedback regime and achieves $\tilde{O}(d\sqrt{T})$ regret under standard linear contextual assumptions. We extend the framework with a Largest-Latency-First (LLF) confidence gate that selectively queries labels, achieving a budget-aware regret bound of $\tilde{O}(\sqrt{dB} + c\sqrt{B})$ with $B=βT$.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。