arXiv:2603.21559cs.CV2026-03被引 1

通过学习对象对的互动概率,提升弱监督视频场景图生成效果。

Revisiting Weakly-Supervised Video Scene Graph Generation via Pair Affinity Learning

  • 引入可学习的配对亲和度,判断对象对是否可能互动。
  • 在动作基因组数据集上超越多个基线模型,显著提升性能。
  • 适合关注视频理解与关系推理的研究者使用。

弱监督视频场景图生成(WS-VSGG)旨在无需边界框标注、仅依赖稀疏时间标签的情况下,将视频内容解析为结构化的语义三元组,大幅降低标注成本。由于缺乏真实边界框,现有方法依赖通用检测器生成物体候选框,但忽略了与全监督流程的根本差异:全监督检测器隐式过滤非交互对象,而通用检测器会无差别地检测所有可见物体,使关系模型被大量噪声配对淹没。为此,本文提出可学习的配对亲和度机制,估计主语-宾语对之间发生交互的可能性。通过配对亲和度学习与评分(PALS),该亲和度被用于推理阶段排序,并进一步通过配对亲和度调制(PAM)融入上下文推理,使模型能抑制非交互配对,聚焦于有语义关系的组合。为提供更干净的监督信号,还提出关系感知匹配(RAM),利用视觉-语言对齐解决伪标签生成中的类别级模糊问题。在Action Genome数据集上的大量实验表明,本方法在不同基线和骨干网络下均实现显著提升,达到当前最优的弱监督视频场景图生成性能。

原文摘要 · Abstract (English)

Weakly-supervised video scene graph generation (WS-VSGG) aims to parse video content into structured relational triplets without bounding box annotations and with only sparse temporal labeling, significantly reducing annotation costs. Without ground-truth bounding boxes, these methods rely on off-the-shelf detectors to generate object proposals, yet largely overlook a fundamental discrepancy from fullysupervised pipelines. Fully-supervised detectors implicitly filter out noninteractive objects, while off-the-shelf detectors indiscriminately detect all visible objects, overwhelming relation models with noisy pairs.We address this by introducing a learnable pair affinity that estimates the likelihood of interaction between subject-object pairs. Through Pair Affinity Learning and Scoring (PALS), pair affinity is incorporated into inferencetime ranking and further integrated into contextual reasoning through Pair Affinity Modulation (PAM), enabling the model to suppress noninteractive pairs and focus on relationally meaningful ones. To provide cleaner supervision for pair affinity learning, we further propose Relation- Aware Matching (RAM), which leverages vision-language grounding to resolve class-level ambiguity in pseudo-label generation. Extensive experiments on Action Genome demonstrate that our approach consistently yields substantial improvements across different baselines and backbones, achieving state-of-the-art WS-VSGG performance.

视频场景图弱监督关系推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。