提升视频场景图中关系预测的准确率，尤其擅长捕捉稀有关系。
RS-Net: Context-Aware Relation Scoring for Dynamic Scene Graph Generation
- 用空间与时间上下文联合评分物体对的关系重要性
- 在Action Genome数据集上平均召回率显著提升，稀有关系识别更好
- 可无缝接入现有模型，无需改动结构，适合做关系建模的增强模块
动态场景图生成（DSGG）旨在建模视频中物体关系的时序演变。现有方法仅在标注的物体对上训练，缺乏对非相关对的指导，导致推理时难以区分有意义的关系。本文提出关系评分网络（RS-Net），通过空间交互和长程时序上下文联合评估物体对的上下文重要性。该框架包含带可学习上下文标记的空间编码器和聚合视频级信息的时序编码器。生成的关系得分被整合进统一三元组评分机制以提升预测性能。RS-Net可无修改地嵌入现有DSGG模型。在Action Genome数据集上的实验表明，其在多种基线模型上均稳定提升召回率与精确率，尤其在平均召回率上表现突出，有效缓解了关系分布的长尾问题。尽管参数量增加，仍保持高效，优于当前最优方法。
原文摘要 · Abstract (English)
Dynamic Scene Graph Generation (DSGG) models how object relations evolve over time in videos. However, existing methods are trained only on annotated object pairs and lack guidance for non-related pairs, making it difficult to identify meaningful relations during inference. In this paper, we propose Relation Scoring Network (RS-Net), a modular framework that scores the contextual importance of object pairs using both spatial interactions and long-range temporal context. RS-Net consists of a spatial context encoder with learnable context tokens and a temporal encoder that aggregates video-level information. The resulting relation scores are integrated into a unified triplet scoring mechanism to enhance relation prediction. RS-Net can be easily integrated into existing DSGG models without architectural changes. Experiments on the Action Genome dataset show that RS-Net consistently improves both Recall and Precision across diverse baselines, with notable gains in mean Recall, highlighting its ability to address the long-tailed distribution of relations. Despite the increased number of parameters, RS-Net maintains competitive efficiency, achieving superior performance over state-of-the-art methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。