arXiv:2605.09065cs.CVcs.LG2026-05

用分层依赖感知扩散模型,从文本生成更准确的场景图。

Dependency-Aware Discrete Diffusion for Scene Graph Generation

论文配图:Dependency-Aware Discrete Diffusion for Scene Graph Generation
图 1 · 摘自论文原文
  • 分步生成结构与语义,捕捉物体间复杂依赖关系
  • 在标准数据集上优于连续与离散基线模型
  • 适合需要精确组合布局的图像生成任务

场景图(SG)将物体及其关系表示为结构化图,在图像生成、机器人和3D理解中具有应用价值。尽管基于场景图的图像生成比纯文本提示更具组合一致性,但用户通常提供自然语言而非结构化图,因此如何从文本生成场景图成为关键挑战。现有离散扩散方法在生成分子、电路等通用图时表现良好,但难以处理场景图中物体、边和关系间的层次结构与强依赖性。本文提出一种依赖感知、分层约束的离散扩散模型,通过解耦前后向过程中的结构与语义,实现条件依赖建模。推理时采用无需训练的条件采样,生成与文本对齐的场景图。在标准场景图基准上评估,本方法在图结构与布局指标上均优于连续与离散基线模型。将其用于下游图像生成时,相较文本到图像模型显著提升多物体场景下的组合一致性。

原文摘要 · Abstract (English)

Scene graphs (SGs) represent objects and their relationships as structured graphs, enabling applications in image generation, robotics, and 3D understanding. Recent work suggests that conditioning image generation on scene graphs improves compositional fidelity compared to text-only prompting. However, since users typically provide text rather than structured graphs, a key challenge is to generate scene graphs from natural language. Prior work on discrete diffusion has demonstrated success in generating generic graphs such as molecules and circuits, but fails to account for the hierarchical structure and strong dependencies between objects, edges, and relations in scene graphs. We address this limitation by introducing a dependency-aware, hierarchically constrained discrete diffusion model for scene graph generation. Our approach decouples structure and semantics across the forward and reverse processes, enabling the model to capture conditional dependencies. At inference time, we perform training-free conditioning to sample text-aligned scene graphs. We evaluate our method on standard SG benchmarks and demonstrate improvements over both continuous and discrete graph generation baselines across graph and layout metrics. When fed to downstream image generation, our approach yields improved compositional alignment compared to text-to-image models, particularly in multi-object scenarios.

场景图生成扩散模型依赖建模

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。