提升视觉语言模型的组合推理能力，通过结构化关系分析解决语义歧义问题。
Inference-Time Structural Reasoning for Compositional Vision-Language Understanding
- 基于依赖解析提取图像文本中的主谓宾三元组，构建场景图增强语义结构。
- Qwen3-VL-8B-Thinking 模型在组合推理任务上达到66.0分，超越现有开源模型。
- 适用于需要精准理解复杂场景关系的视觉问答与内容生成任务。
视觉语言模型（VLMs）在图像-文本检索任务中表现优异，但在组合推理方面仍存在缺陷，难以区分词语相同但关系结构不同的描述。本文提出一个统一评估与增强框架，对四种架构各异的VLMs（CLIP、BLIP、LLaVA、Qwen3-VL-8B-Thinking）在无修饰和场景图增强两种条件下进行评估。引入基于依赖关系的TextSceneGraphParser（spaCy）提取主-关系-宾三元组，并设计图不对称评分器，利用最优二分匹配注入结构先验。通过主体-客体掩码与互换消融实验发现，Qwen3-VL-8B-Thinking 在组合推理任务中取得62.75的群体得分，显著优于所有编码器类模型；进一步采用多轮场景图过滤策略后，得分提升至66.0，超越现有开源最先进水平。分析表明，场景图增强对已有较强能力的模型有显著增益，而对基础较弱模型则效果有限或为负。代码已开源。
原文摘要 · Abstract (English)
Vision-language models (VLMs) excel at image-text retrieval yet persistently fail at compositional reasoning, distinguishing captions that share the same words but differ in relational structure. We present, a unified evaluation and augmentation framework benchmarking four architecturally diverse VLMs,CLIP, BLIP, LLaVA, and Qwen3-VL-8B-Thinking,on the Winoground benchmark under plain and scene-graph-augmented regimes. We introduce a dependency-based TextSceneGraphParser (spaCy) extracting subject-relation-object triples, and a Graph Asymmetry Scorer using optimal bipartite matching to inject structural relational priors. Caption ablation experiments (subject-object masking and swapping) reveal that Qwen3-VL-8B-Thinking achieves a group score of 62.75, far above all encoder-based models, while a proposed multi-turn SG filtering strategy further lifts it to 66.0, surpassing prior open-source state-of-the-art. We analyze the capability augmentation tradeoff and find that SG augmentation benefits already capable models while providing negligible or negative gains for weaker baselines. Code: https://github.com/amartyacodes/Inference-Time-Structural-Reasoning-for-Compositional-Vision-Language-Understanding
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。