通过分步诊断与标记生成，让大模型写文本更准长度。
From Sub-Ability Diagnosis to Human-Aligned Generation: Bridging the Gap for Text Length Control via MARKERGEN
- 拆解文本长度控制的子能力，用人工模式定位问题
- 引入动态标记和外部工具，显著提升长度控制效果
- 三阶段生成策略兼顾长度精准与内容质量，适合实际应用
尽管大型语言模型（LLMs）发展迅速，其可控文本长度生成（LCTG）能力仍不尽如人意，严重制约实际应用。现有方法多依赖端到端训练强化长度约束，但缺乏对LCTG子能力的分解与针对性优化，限制了进一步提升。为此，我们以人类行为模式为参考，自下而上分解LCTG子能力，并进行细致错误分析。在此基础上，提出MarkerGen——一种简单却高效的即插即用方法：(1) 通过外部工具集成缓解大模型基础缺陷；(2) 利用动态插入标记实现显式长度建模；(3) 采用三阶段生成策略，在保持内容质量的同时更好对齐长度约束。全面实验表明，MarkerGen在多种设置下显著提升LCTG性能，表现出卓越的有效性与泛化能力。
原文摘要 · Abstract (English)
Despite the rapid progress of large language models (LLMs), their length-controllable text generation (LCTG) ability remains below expectations, posing a major limitation for practical applications. Existing methods mainly focus on end-to-end training to reinforce adherence to length constraints. However, the lack of decomposition and targeted enhancement of LCTG sub-abilities restricts further progress. To bridge this gap, we conduct a bottom-up decomposition of LCTG sub-abilities with human patterns as reference and perform a detailed error analysis. On this basis, we propose MarkerGen, a simple-yet-effective plug-and-play approach that:(1) mitigates LLM fundamental deficiencies via external tool integration;(2) conducts explicit length modeling with dynamically inserted markers;(3) employs a three-stage generation scheme to better align length constraints while maintaining content quality. Comprehensive experiments demonstrate that MarkerGen significantly improves LCTG across various settings, exhibiting outstanding effectiveness and generalizability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。