让扩散模型更准确地数清指定数量的物体，无需重新训练。
CountSteer: Steering Attention for Object Counting in Diffusion Models
- 通过调整交叉注意力隐藏状态，引导生成过程中的计数行为。
- 在不损失图像质量的前提下，计数准确率提升约4%。
- 适合需要精确控制物体数量的文本生成场景。
文本到图像的扩散模型虽能生成逼真连贯的图像，但常忽略文本中的数量指令，暴露出语言与视觉表征之间的差距。有趣的是，我们发现这些模型并非完全无视数字——它们内部信号会因输出是否符合指定数量而呈现一致变化，表明模型已隐式编码了计数正确性的潜在概念。基于此直觉，我们提出 CountSteer，一种无需训练的方法，在推理阶段通过调节模型的交叉注意力隐藏状态来提升指定物体数量的生成准确性。实验表明，CountSteer 在不损害图像质量的情况下，将计数准确率提升了约4%，为实现更可控、语义可靠的文本到图像生成提供了一种简单而有效的方法。
原文摘要 · Abstract (English)
Text-to-image diffusion models generate realistic and coherent images but often fail to follow numerical instructions in text, revealing a gap between language and visual representation. Interestingly, we found that these models are not entirely blind to numbers-they are implicitly aware of their own counting accuracy, as their internal signals shift in consistent ways depending on whether the output meets the specified count. This observation suggests that the model already encodes a latent notion of numerical correctness, which can be harnessed to guide generation more precisely. Building on this intuition, we introduce CountSteer, a training-free method that improves generation of specified object counts by steering the model's cross-attention hidden states during inference. In our experiments, CountSteer improved object-count accuracy by about 4% without compromising visual quality, demonstrating a simple yet effective step toward more controllable and semantically reliable text-to-image generation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。