用特殊标记控制大模型输出长度，不降低生成质量。
Hansel: Output Length Controlling Framework for Large Language Models
- 通过周期性插入特殊令牌追踪剩余输出长度。
- 在4个模型上平均误差显著降低，长/短文本生成更精准。
- 无需修改模型结构，适合各类预训练模型微调使用。
尽管大型语言模型（LLMs）取得巨大成功，但高效控制输出序列长度仍是挑战。本文提出Hansel框架，可在不损害生成能力的前提下实现高效的长度控制。该方法通过周期性输出的隐藏特殊令牌来跟踪目标输出长度。结合避免输出突然终止的技术，此简单策略表现出高效与通用性，同时保持生成文本的连贯性与流畅性。该框架可应用于任意预训练的LLM在微调阶段，不受原始位置编码方式限制。我们在四个不同模型上使用Hansel进行微调，结果显示，在所有模型和数据集上，输出序列的平均绝对误差均显著低于基于提示的长度控制微调方法。此外，该框架在未见目标长度（如长对话回复或极短摘要）上展现出明显外推能力，表明模型学习的是通用长度控制机制，而非仅记忆训练中见过的长度。
原文摘要 · Abstract (English)
Despite the great success of large language models (LLMs), efficiently controlling the length of the output sequence still remains a challenge. In this paper, we propose Hansel, an efficient framework for length control in LLMs without affecting its generation ability. Hansel utilizes periodically outputted hidden special tokens to keep track of the remaining target length of the output sequence. Together with techniques to avoid abrupt termination of the output, this seemingly simple method proved to be efficient and versatile, while not harming the coherency and fluency of the generated text. The framework can be applied to any pre-trained LLMs during the finetuning stage of the model, regardless of its original positional encoding method. We demonstrate this by finetuning four different LLMs with Hansel and show that the mean absolute error of the output sequence decreases significantly in every model and dataset compared to the prompt-based length control finetuning. Moreover, the framework showed a substantially improved ability to extrapolate to target lengths unseen during finetuning, such as long dialog responses or extremely short summaries. This indicates that the model learns the general means of length control, rather than learning to match output lengths to those seen during training.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。