让大模型精准控制输出长度，误差小于3个词元。
Precise Length Control in Large Language Models
- 在输入嵌入中加入倒计时位置编码，引导模型按指定长度停止生成。
- 在问答和摘要任务中实现平均误差低于3个词元的精确长度控制。
- 适合需要固定输出长度的应用，如接口调用或结构化文本生成。
大型语言模型（LLMs）在生产系统中广泛应用，支撑聊天机器人、摘要生成和问答等任务。尽管成果显著，但控制输出长度仍是难题，尤其在需要结构化输出或特定细节层次的任务中。本文提出一种方法，用于调整预训练的仅解码器架构大模型，实现对响应长度的精确控制。该方法在输入嵌入中引入一种次级长度差分位置编码（LDPE），可倒数至用户设定的终止长度。通过带有LDPE的微调，模型能学会在期望长度处连贯地终止生成，平均词元误差低于3个。我们还提出了Max New Tokens++，支持灵活的上限长度控制而非精确目标。在问答和文档摘要任务上的实验表明，该方法可在不牺牲生成质量的前提下实现精确长度控制。
原文摘要 · Abstract (English)
Large Language Models (LLMs) are increasingly used in production systems, powering applications such as chatbots, summarization, and question answering. Despite their success, controlling the length of their response remains a significant challenge, particularly for tasks requiring structured outputs or specific levels of detail. In this work, we propose a method to adapt pre-trained decoder-only LLMs for precise control of response length. Our approach incorporates a secondary length-difference positional encoding (LDPE) into the input embeddings, which counts down to a user-set response termination length. Fine-tuning with LDPE allows the model to learn to terminate responses coherently at the desired length, achieving mean token errors of less than 3 tokens. We also introduce Max New Tokens++, an extension that enables flexible upper-bound length control, rather than an exact target. Experimental results on tasks such as question answering and document summarization demonstrate that our method enables precise length control without compromising response quality.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。