让大模型精准生成带45个细粒度属性的文本,效果远超现有方法。
UltraGen: Extremely Fine-grained Controllable Generation via Attribute Reconstruction and Global Preference Optimization
- 用软硬属性结合生成45组精细要求,指导文本重建。
- 通过偏好优化提升多属性组合下的生成质量与约束满足率。
- 适合需要高精度控制文本风格、结构和格式的研究者。
细粒度可控文本生成是大模型的重要需求,但现有方法通常仅支持3至5个属性,当属性数量增至约45个时性能急剧下降。为此,我们提出一种零样本的极端细粒度可控生成(EFCG)框架,包含自重构(AR)与全局偏好优化(GPO)两阶段。在AR阶段,利用大模型从原始文本中提取软属性(如设计上强调简洁与极简),结合程序生成的硬属性(如字数在300至400之间),构建约45个多属性要求,指导弱监督下的细粒度文本重构。在GPO阶段,采用直接偏好优化(DPO)对多样化属性组合进行优化,高效探索全局组合空间。此外,引入高效的属性采样策略以识别并修正潜在错误属性,进一步提升全局优化效果。该框架显著提升了约束满足率(CSR)与文本质量,有效缓解位置偏差与注意力稀释问题。
原文摘要 · Abstract (English)
Fine granularity is an essential requirement for controllable text generation, which has seen rapid growth with the ability of LLMs. However, existing methods focus mainly on a small set of attributes like 3 to 5, and their performance degrades significantly when the number of attributes increases to the next order of magnitude. To address this challenge, we propose a novel zero-shot approach for extremely fine-grained controllable generation (EFCG), proposing auto-reconstruction (AR) and global preference optimization (GPO). In the AR phase, we leverage LLMs to extract soft attributes (e.g., Emphasis on simplicity and minimalism in design) from raw texts, and combine them with programmatically derived hard attributes (e.g., The text should be between 300 and 400 words) to construct massive (around 45) multi-attribute requirements, which guide the fine-grained text reconstruction process under weak supervision. In the GPO phase, we apply direct preference optimization (DPO) to refine text generation under diverse attribute combinations, enabling efficient exploration of the global combination space. Additionally, we introduce an efficient attribute sampling strategy to identify and correct potentially erroneous attributes, further improving global optimization. Our framework significantly improves the constraint satisfaction rate (CSR) and text quality for EFCG by mitigating position bias and alleviating attention dilution.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。