让视频生成更准数，不用训练就能纠正提示词中的数量错误
When Numbers Speak: Aligning Textual Numerals and Visual Instances in Text-to-Video Diffusion Models
- 通过注意力机制识别提示词与画面的数字不匹配
- 在5B和14B模型上计数准确率提升4.9%至5.5%
- 适合关注生成内容数量准确性的人使用
文本到视频的扩散模型虽能实现开放式的视频生成，但常无法正确生成提示词中指定数量的物体。本文提出NUMINA，一种无需训练的“识别-引导”框架，以改善数量对齐。NUMINA通过选择具有判别性的自注意力和交叉注意力头，提取可计数的潜在布局，识别提示词与画面间的不一致；随后保守地优化该布局，并调制交叉注意力以引导重生成。在新提出的CountBench基准上，NUMINA在Wan2.1-1.3B模型上将计数准确率提升7.4%，在5B和14B模型上分别提升4.9%和5.5%。同时，CLIP对齐性能提升，且保持时间一致性。结果表明，结构化引导可补充种子搜索与提示增强，为实现计数准确的文本到视频扩散模型提供了一条实用路径。代码已开源。
原文摘要 · Abstract (English)
Text-to-video diffusion models have enabled open-ended video synthesis, but often struggle with generating the correct number of objects specified in a prompt. We introduce NUMINA , a training-free identify-then-guide framework for improved numerical alignment. NUMINA identifies prompt-layout inconsistencies by selecting discriminative self- and cross-attention heads to derive a countable latent layout. It then refines this layout conservatively and modulates cross-attention to guide regeneration. On the introduced CountBench, NUMINA improves counting accuracy by up to 7.4% on Wan2.1-1.3B, and by 4.9% and 5.5% on 5B and 14B models, respectively. Furthermore, CLIP alignment is improved while maintaining temporal consistency. These results demonstrate that structural guidance complements seed search and prompt enhancement, offering a practical path toward count-accurate text-to-video diffusion. The code is available at https://github.com/H-EmbodVis/NUMINA.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。