用简单偏置调整模型输出，无需微调就能提升特定任务表现。
Frustratingly Simple Black-Box Adaptation of Language Models via Logit Bias
- 通过在解码时添加固定偏置向量，不改模型权重实现适应。
- 在数学和推理任务上超越基础模型，参数量远少于微调方法。
- 适合资源有限、无法微调但需定制化输出的场景。
许多机构希望将语言模型适配到内部使用，以提升特定任务性能并解决敏感数据隐私问题。然而，这种适配仍具挑战性：通常需要操作复杂的开源模型微调或临时提示优化。本文研究一种极简替代方案，即通过用户定义的向量在API层面控制模型的logits。我们提出一种黑盒方法，学习一个上下文无关的固定logit偏置向量，在每一步解码时添加，无需修改模型权重或梯度。基于KL正则化的强化学习目标，我们推导出该固定偏置可近似最优前缀依赖修正的条件，并从采样轨迹、奖励和词元概率中得出闭式反倾向估计器。实验表明，这种仅在解码阶段干预的方法，在数学与推理基准上优于基础模型，且可训练参数远少于传统微调。结果表明，学习到的logit偏置是一种在最小访问条件下适配语言模型的轻量机制。
原文摘要 · Abstract (English)
Many organizations aim to adapt language models for internal use, both to improve performance on domain-specific tasks and to address privacy concerns around sensitive data. However, such adaptation remains non-trivial: it often requires operationally challenging fine-tuning of open-source models or ad hoc prompt optimization. We study a minimal alternative based on a simple API-level control: allowing users to bias the model's logits with a user-defined vector. We develop a black-box method for learning a single context-independent logit-bias vector, added at every decoding step, without modifying model weights or requiring gradients. Starting from a KL-regularized reinforcement learning (RL) objective, we characterize when such a fixed logit-bias vector can approximate the optimal prefix-dependent correction and derive a closed-form inverse-propensity estimator from rollouts, rewards, and token probabilities. Empirically, this simple decoding-time intervention improves over base models on mathematical and reasoning benchmarks while using far fewer trainable parameters than conventional fine-tuning. Our results suggest that learned logit bias is a lightweight mechanism for adapting language models under minimal access requirements.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。