通过多跳关系对比学习，让模型理解物体间更复杂的空间关系。
Multi-hop Relational Contrastive Learning: Extending Spatial Contrastive Pre-training Beyond Pairwise Relations
- 构建场景图并追踪多跳路径，捕捉间接空间依赖
- 在GQA子集上提升内容检索准确率至NDCG@5=0.748
- 适合需要理解复杂空间结构的视觉任务
理解物体间的空间关系是场景理解的基础，但现有对比预训练方法仅建模成对关系，忽视了更丰富的组合与多跳交互。本文提出多跳关系对比学习（MRCL），将空间对比学习扩展至图结构场景表示。通过在由检测物体构建的场景图中追踪k跳路径，MRCL捕获超越直接对象对表达能力的隐含空间依赖。定义涵盖节点、边及多跳路径的多层次对比目标，使嵌入在保持语义稳定性的同时响应空间布局变化。在GQA子集上，MRCL生成的空间感知表示显著提升基于内容的图检索性能（NDCG@5 = 0.748），并持续改善下游任务表现，包括空间关系识别与基于图的问题回答。结果表明，相比仅成对监督，多跳关系监督提供更丰富的结构引导，生成更具鲁棒性、组合性与几何感知性的视觉表征。
原文摘要 · Abstract (English)
Understanding how objects relate to each other in space is fundamental to scene understanding, yet most contrastive pre-training approaches only model pairwise relationships, leaving richer compositional and multi-hop interactions largely unexplored. We introduce Multi-Hop Relational Contrastive Learning (MRCL), a framework that extends spatial contrastive learning to graph-structured scene representations. By tracing k-hop paths through scene graphs built from detected objects, MRCL captures implicit spatial dependencies that go well beyond what direct object pairs can express. We define a multi-level contrastive objective spanning nodes, edges, and multi-hop paths, encouraging embeddings that remain stable across object semantics while staying responsive to spatial layout. On a GQA subset, MRCL produces spatially-aware representations that improve content-based graph retrieval (NDCG@5 = 0.748) and consistently benefit downstream tasks, including spatial relationship recognition and graph-based question answering. Together, these results suggest that multi-hop relational supervision offers substantially richer structural guidance than pairwise-only methods, leading to visual representations that are more robust, compositional, and geometry-aware.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。