LingGen实现40个语言属性的精细控制，效果优于现有方法。
LingGen: Scalable Multi-Attribute Linguistic Control via Power-Law Masking
- 用帕累托分布采样掩码率，提升多属性控制鲁棒性。
- 在1到40个属性控制下，平均误差最低且生成流畅度最高。
- 适合需要高精度语言风格调控的研究与应用。
我们提出LingGen，一种支持对大量实值语言属性进行细粒度控制的文本生成模型。它通过专用的语言属性编码器编码目标属性值，并将生成的表征注入语言模型的序列起始（BOS）嵌入中以实现条件生成。为提升对不同属性子集控制的鲁棒性，引入P-MASKING，在训练时从截断的帕累托分布中为每个样本采样属性掩码率。在1至40个控制属性的实验中，LingGen在对比方法中实现了最低的平均控制误差，同时推理效率高，人类评估得分最高的流畅度。消融实验表明，帕累托采样的掩码策略和基于BOS的注入方式相比其他变体更有效。
原文摘要 · Abstract (English)
We present LingGen, a controlled text generation model that allows fine-grained control over a large number of real-valued linguistic attributes. It encodes target attribute values with a dedicated linguistic attribute encoder and conditions the language model by injecting the resulting representation into the language model using the beginning-of-sequence (BOS) embeddings. To improve robustness when controlling different attribute subsets, we introduce P-MASKING, which samples per-example attribute masking rates from a truncated Pareto distribution during training. Across 1-40 control attributes, LingGen achieves the lowest average control error among evaluated methods, while remaining efficient at inference and receiving the highest fluency scores in human evaluation. Ablations show that Pareto-sampled masking and BOS-based injection are effective choices compared to alternative masking and integration variants.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。