提出连续进度比嵌入，实现稳定可控的文本生成长度。
Progress Ratio Embeddings: An Impatience Signal for Robust Length Control in Neural Text Generation
- 用三角函数生成的连续信号替代离散倒计时，提升长度控制稳定性。
- 在两个新闻摘要数据集上，对未见长度目标仍保持高精度控制。
- 无需牺牲生成质量，适合需要精准长度输出的场景。
现代神经语言模型在文本生成中表现优异，但对生成长度的精确控制仍不成熟。本文首先分析基于反向位置嵌入（RPE）的近期长度控制方法，发现其在超出训练分布的长度控制任务中存在局限性。具体表现为：依赖绝对剩余词数的离散倒计时信号会导致不稳定。为此，我们提出进度比嵌入（Progress Ratio Embeddings, PRE），通过连续的三角函数型急切信号构建嵌入。PRE可无缝集成至标准Transformer架构，在不降低标准评估指标下实现稳定的长度保真度。实验表明，PRE在两个主流新闻摘要基准上均具备良好的泛化能力，适用于未见过的目标长度。
原文摘要 · Abstract (English)
Modern neural language models achieve high accuracy in text generation, yet precise control over generation length remains underdeveloped. In this paper, we first investigate a recent length control method based on Reverse Positional Embeddings (RPE) and show its limits when control is requested beyond the training distribution. In particular, using a discrete countdown signal tied to the absolute remaining token count leads to instability. To provide robust length control, we introduce Progress Ratio Embeddings (PRE), as continuous embeddings tied to a trigonometric impatience signal. PRE integrates seamlessly into standard Transformer architectures, providing stable length fidelity without degrading text accuracy under standard evaluation metrics. We further show that PRE generalizes well to unseen target lengths. Experiments on two widely used news-summarization benchmarks validate these findings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。