根据任务需求动态调整采样温度，提升模型生成效果。
Adaptive Decoding via Latent Preference Optimization
- 在推理时动态选择采样温度，支持逐词或逐例调整。
- 在UltraFeedback、创意写作和GSM8K上超越所有固定温度设置。
- 适用于需要平衡创意与准确性的各类语言任务。
语言模型解码中，高温度采样更富创意，低温度更准确。但现有模型通常对所有任务统一使用固定温度。本文提出自适应解码，在推理时动态选择采样温度，可在词级或例级进行调整，以优化性能。为此引入隐变量偏好优化（LPO），一种训练离散隐变量（如温度选择）的通用方法。实验表明，该方法在需不同温度的任务中表现优异，涵盖UltraFeedback、创意故事生成和GSM8K等数据集。
原文摘要 · Abstract (English)
During language model decoding, it is known that using higher temperature sampling gives more creative responses, while lower temperatures are more factually accurate. However, such models are commonly applied to general instruction following, which involves both creative and fact seeking tasks, using a single fixed temperature across all examples and tokens. In this work, we introduce Adaptive Decoding, a layer added to the model to select the sampling temperature dynamically at inference time, at either the token or example level, in order to optimize performance. To learn its parameters we introduce Latent Preference Optimization (LPO) a general approach to train discrete latent variables such as choices of temperature. Our method outperforms all fixed decoding temperatures across a range of tasks that require different temperatures, including UltraFeedback, Creative Story Writing, and GSM8K.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。