arXiv:2510.19278cs.CV2025-10被引 1

让检测模型当计数裁判,提升文生图的数字准确性

D2D: Detector-to-Differentiable Critic for Improved Numeracy in Text-to-Image Generation

  • 用自定义激活函数把检测器输出转为可微信号,实现非可微模型的梯度指导
  • 在多个数据集上提升物体数量生成准确率,最高达13.7%且图像质量几乎不变
  • 适合关注文生图中数字精准性的研究者和应用开发者

文生图扩散模型在语义对齐上表现良好,但在按提示词生成正确物体数量方面仍存在困难。现有方法通常依赖外部可微的计数网络作为辅助判别器,但受限于梯度传播需求,无法使用计数能力更强的检测类模型(因其计数基于枚举,不可微)。为此,本文提出 Detector-to-Differentiable (D2D) 框架,将非可微检测模型转化为可微判别器,从而利用其优越的计数能力指导生成过程。具体地,设计定制激活函数将检测器置信度转为软二值指示,用于优化预训练文生图模型推理时的噪声先验。在 SDXL-Turbo、SD-Turbo 与 Pixart-DMD 上,针对四个不同复杂度基准(低密度、高密度、多物体场景)的实验表明,该方法在对象计数准确率上实现持续显著提升(如在 400 提示词的低密度基准 D2D-Small 上提升高达 13.7%),同时保持图像质量稳定且计算开销极小。

原文摘要 · Abstract (English)

Text-to-image (T2I) diffusion models have achieved strong performance in semantic alignment, yet they still struggle with generating the correct number of objects specified in prompts. Existing approaches typically incorporate auxiliary counting networks as external critics to enhance numeracy. However, since these critics must provide gradient guidance during generation, they are restricted to regression-based models that are inherently differentiable, thus excluding detector-based models with superior counting ability, whose count-via-enumeration nature is non-differentiable. To overcome this limitation, we propose Detector-to-Differentiable (D2D), a novel framework that transforms non-differentiable detection models into differentiable critics, thereby leveraging their superior counting ability to guide numeracy generation. Specifically, we design custom activation functions to convert detector logits into soft binary indicators, which are then used to optimize the noise prior at inference time with pre-trained T2I models. Our extensive experiments on SDXL-Turbo, SD-Turbo, and Pixart-DMD across four benchmarks of varying complexity (low-density, high-density, and multi-object scenarios) demonstrate consistent and substantial improvements in object counting accuracy (e.g., boosting up to 13.7% on D2D-Small, a 400-prompt, low-density benchmark), with minimal degradation in overall image quality and computational overhead.

文生图计数准确扩散模型可微化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。