让大模型按需生成指定长度文本，提升实用性和可控性。
Controlled Diversity: Length-optimized Natural Language Generation
- 用数据增强与微调训练模型，使其响应长度更符合要求。
- 模型生成文本长度更贴近目标，且整体质量优于基线。
- 使用自生成数据训练可避免质量下降，适合对齐多目标场景。
大语言模型通常无法根据严格的长度要求调整输出，而这一能力在需要满足多样用户与系统需求的应用中至关重要。本文提出一种方法，通过扩充现有数据并应用现有微调技术，训练大模型实现长度可控生成。我们对比了不同训练策略下模型对长度要求的遵循程度及整体响应质量。结果表明，该方法能有效使模型生成更符合长度要求的文本；但若训练数据非基线模型生成，则可能影响响应质量。使用模型自生成数据进行训练可消除此问题，在特定场景下可同时对齐其他训练目标，否则则不推荐。
原文摘要 · Abstract (English)
LLMs are not generally able to adjust the length of their outputs based on strict length requirements, a capability that would improve their usefulness in applications that require adherence to diverse user and system requirements. We present an approach to train LLMs to acquire this capability by augmenting existing data and applying existing fine-tuning techniques, which we compare based on the trained models' adherence to the length requirement and overall response quality relative to the baseline model. Our results demonstrate that these techniques can be successfully applied to train LLMs to adhere to length requirements, with the trained models generating texts which better align to the length requirements. Our results indicate that our method may change the response quality when using training data that was not generated by the baseline model. This allows simultaneous alignment to another training objective in certain scenarios, but is undesirable otherwise. Training on a dataset containing the model's own responses eliminates this issue.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。