用概率理论优化多属性文本生成，让模型像调色盘一样精准控制输出风格。
Palette of Language Models: A Solver for Controlled Text Generation
- 基于概率原理设计新组合策略，避免多属性冲突。
- 在单属性和多属性场景下均超越现有方法效果。
- 适合需要精细控制文本风格的研究者与开发者。
大语言模型在文本生成方面取得了显著进展，能通过恰当提示生成符合特定要求的文本。然而，同时控制多个属性时设计最优提示仍具挑战性。传统线性组合单一属性模型的方法常忽略属性间重叠，导致冲突。为此，我们提出一种受全概率定律与条件互信息最小化启发的新组合策略。该方法被命名为「语言模型调色板」，因其将属性强度与生成风格的关联类比为调色过程。此外，我们提出正相关性和属性增强作为指导合理组合设计的理论性质。在单属性与多属性控制任务中进行实验，结果均优于现有方法。
原文摘要 · Abstract (English)
Recent advancements in large language models have revolutionized text generation with their remarkable capabilities. These models can produce controlled texts that closely adhere to specific requirements when prompted appropriately. However, designing an optimal prompt to control multiple attributes simultaneously can be challenging. A common approach is to linearly combine single-attribute models, but this strategy often overlooks attribute overlaps and can lead to conflicts. Therefore, we propose a novel combination strategy inspired by the Law of Total Probability and Conditional Mutual Information Minimization on generative language models. This method has been adapted for single-attribute control scenario and is termed the Palette of Language Models due to its theoretical linkage between attribute strength and generation style, akin to blending colors on an artist's palette. Moreover, positive correlation and attribute enhancement are advanced as theoretical properties to guide a rational combination strategy design. We conduct experiments on both single control and multiple control settings, and achieve surpassing results.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。