用实例比调参数更能突破生成模型的控制极限。
The Steering Budget: Examples beat Knobs
- 用训练数据预估控制预算，区分旋钮可调范围与实例可及范围。
- 实例方法可达旋钮无法触及的更大控制区间，提升生成灵活性。
- 适用于无法用语言描述的目标，适合复杂创意生成任务。
生成模型通过旋钮（如提示词、引导系数）进行调控，但超过某阈值后效果饱和。我们发现这并非模型缺陷，而是由训练数据决定的控制预算：属性的可调范围分为两部分——旋钮能触及的有限部分，以及更大的、仅能通过具体实例实现的部分。后者通常更显著，且可通过分析训练数据提前判断。使用模型已学知识构建的实例集，可突破旋钮限制，实现全范围控制。该方法兼具更强可达性与表达力，支持仅能通过样例描述的目标（包括不可言说的）。我们在图像与晶体结构生成两个独立领域验证了这一机制，明确划分了旋钮足够与必须依赖实例的场景。
原文摘要 · Abstract (English)
Generative models are steered with knobs -- prompts, guidance scales, property tags. Turn one as hard as you like and, past a point, it stops moving the property you care about. We find that ceiling is not a shortcoming of the model but a budget, set by the training data before the model is trained: a property's movable range splits in two -- the part a knob can reach, and a second, significant part that only examples -- concrete instances of what you want more of -- can reach. That second part is usually much larger, but not always, and the same budget says so in advance. Reaching that second part takes a different move: instead of turning a knob, you show the model examples, composed from what it already learned rather than added to its training. A cheap audit of the training data measures the budget; we give a recipe for building the example set that reaches all of it. This buys two things a knob can't. Reach: it moves a property across the whole budget, not just the part a knob reaches. Expressiveness: it steers toward targets you can only specify by example -- including ones you can't put into words. We turn these into a handful of falsifiable claims and verify them in two unrelated domains, image and crystal-structure generation -- marking where a knob is enough, and where only examples will do.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。