arXiv:2509.16141cs.CV2025-09EMNLP

提升文本生成图像中动作描绘的准确性,解决细节缺失问题。

AcT2I: Evaluating and Improving Action Depiction in Text-to-Image Models

  • 用大语言模型增强提示词,注入时间等细粒度信息
  • 改进后模型在动作图像生成上准确率提升72%
  • 适合关注动作理解与场景细节生成的研究者

文本到图像(T2I)模型在生成基于文本描述的图像方面取得了显著进展,但在以动作和交互为核心语义的复杂场景中仍存在挑战。本文观察到,现有T2I模型难以捕捉动作描述中隐含的细微属性,导致生成图像缺少关键上下文细节。为此,我们提出了AcT2I基准,用于系统评估T2I模型在动作导向提示下的表现。实验表明,主流T2I模型在AcT2I上表现不佳。我们进一步提出一种无需训练的知识蒸馏方法,利用大语言模型增强提示词,在时间、主体、环境三个维度注入密集信息。结果显示,加入时间信息显著提升生成准确性,最佳模型提升达72%。研究揭示了当前T2I方法在复杂推理任务中的局限性,并证明系统性融合语言知识可有效提升图像生成的细腻度与上下文准确性。

原文摘要 · Abstract (English)

Text-to-Image (T2I) models have recently achieved remarkable success in generating images from textual descriptions. However, challenges still persist in accurately rendering complex scenes where actions and interactions form the primary semantic focus. Our key observation in this work is that T2I models frequently struggle to capture nuanced and often implicit attributes inherent in action depiction, leading to generating images that lack key contextual details. To enable systematic evaluation, we introduce AcT2I, a benchmark designed to evaluate the performance of T2I models in generating images from action-centric prompts. We experimentally validate that leading T2I models do not fare well on AcT2I. We further hypothesize that this shortcoming arises from the incomplete representation of the inherent attributes and contextual dependencies in the training corpora of existing T2I models. We build upon this by developing a training-free, knowledge distillation technique utilizing Large Language Models to address this limitation. Specifically, we enhance prompts by incorporating dense information across three dimensions, observing that injecting prompts with temporal details significantly improves image generation accuracy, with our best model achieving an increase of 72%. Our findings highlight the limitations of current T2I methods in generating images that require complex reasoning and demonstrate that integrating linguistic knowledge in a systematic way can notably advance the generation of nuanced and contextually accurate images.

文本生成图像动作识别提示工程语言模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。