arXiv:2512.23024cs.CV2025-12

用场景图结构显式建模物体上下文,提升识别准确率。

With Great Context Comes Great Prediction Power: Classifying Objects via Geo-Semantic Scene Graphs

论文配图:With Great Context Comes Great Prediction Power: Classifying Objects via Geo-Semantic Scene Graphs
图 1 · 摘自论文原文
  • 从单目图像构建包含几何、材质和空间关系的场景图
  • 准确率达73.4%,远超无上下文模型(最低38.4%)
  • 结果可解释,适合需要推理透明性的应用场景

人类通过理解周围场景中的空间关系、材质属性及物体共现模式来轻松识别物体。相比之下,大多数计算对象识别系统仅处理孤立图像区域,忽视了关键上下文信息。本文强调上下文的重要性,提出一种新的上下文对象分类框架。首先,从单目图像构建地理语义上下文图(GSCG),通过融合度量深度估计与统一的全景及材质分割模型生成。GSCG将物体表示为带有几何、色彩和材质属性的节点,空间关系作为边,形成显式结构,使模型推理过程天然可解释。随后设计专用图分类器,聚合目标物体、邻近物体及全局场景特征进行分类。大量消融实验表明,该上下文感知模型在COCO 2017训练/验证集上达到73.4%的分类准确率,显著优于无上下文版本(最低38.4%)。此外,其性能大幅超越强基线方法,包括微调后的ResNet(最高53.5%)和先进的多模态大语言模型Llama 4 Scout(即使结合完整图像与详细描述,最高仅42.3%)。这些结果凸显显式结构化且可解释上下文在对象识别任务中的优势。

原文摘要 · Abstract (English)

Humans effortlessly identify objects by leveraging a rich understanding of the surrounding scene, including spatial relationships, material properties, and the co-occurrence of other objects. In contrast, most computational object recognition systems operate on isolated image regions, devoid of meaning in isolation, thus ignoring this vital contextual information. This paper argues for the critical role of context and introduces a novel framework for contextual object classification. We first construct a Geo-Semantic Contextual Graph (GSCG) from a single monocular image. This rich, structured representation is built by integrating a metric depth estimator with a unified panoptic and material segmentation model. The GSCG encodes objects as nodes with detailed geometric, chromatic, and material attributes, and their spatial relationships as edges. This explicit graph structure makes the model's reasoning process inherently interpretable. We then propose a specialized graph-based classifier that aggregates features from a target object, its immediate neighbors, and the global scene context to predict its class. Through extensive ablation studies, we demonstrate that our context-aware model achieves a classification accuracy of 73.4%, dramatically outperforming context-agnostic versions (as low as 38.4%). Furthermore, our GSCG-based approach significantly surpasses strong baselines, including fine-tuned ResNet models (max 53.5%) and a state-of-the-art multimodal Large Language Model (LLM), Llama 4 Scout, which, even when given the full image alongside a detailed description of objects, maxes out at 42.3%. These results on COCO 2017 train/val splits highlight the superiority of explicitly structured and interpretable context for object recognition tasks.

场景图上下文建模可解释性图像识别

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。