提出混合关系分配机制,提升单阶段场景图生成的准确率
Hydra-SGG: Hybrid Relation Assignment for One-stage Scene Graph Generation
- 采用一对一与基于IoU的多对一结合的关系分配策略
- 在VG150、Open Images V6等数据集上达最新性能
- 无自注意力的辅助分支促进多样预测,缓解稀疏标注问题
DETR引入了一种简化的单阶段场景图生成(SGG)框架,但面临监督稀疏和误负样本的问题。前者源于每张图像通常仅有不到10个关系标注,而基于DETR的SGG模型使用超过100个关系查询;每个真实关系仅分配给一个查询。后者则因一个真实关系可能对应多个匹配分数相近的查询,导致次优查询被误判为负样本。为此,我们提出Hydra-SGG,一种结合一对一与基于IoU的多对一关系分配的单阶段SGG方法,增加正样本数量并缓解稀疏监督问题。此外,我们实证发现移除关系查询间的自注意力会引发重复预测,这反而有助于多对一分配。基于此,我们设计了无自注意力层的Hydra Branch辅助解码器,进一步增强多对一分配能力,促使不同查询做出相同关系预测。Hydra-SGG在多个数据集上达到领先性能:VG150(mR@50为16.0)、Open Images V6(加权得分50.1)、GQA(mR@50为12.7)。
原文摘要 · Abstract (English)
DETR introduces a simplified one-stage framework for scene graph generation (SGG) but faces challenges of sparse supervision and false negative samples. The former occurs because each image typically contains fewer than 10 relation annotations, while DETR-based SGG models employ over 100 relation queries. Each ground truth relation is assigned to only one query during training. The latter arises when one ground truth relation may have multiple queries with similar matching scores, leading to suboptimally matched queries being treated as negative samples. To address these, we propose Hydra-SGG, a one-stage SGG method featuring a Hybrid Relation Assignment. This approach combines a One-to-One Relation Assignment with an IoU-based One-to-Many Relation Assignment, increasing positive training samples and mitigating sparse supervision. In addition, we empirically demonstrate that removing self-attention between relation queries leads to duplicate predictions, which actually benefits the proposed One-to-Many Relation Assignment. With this insight, we introduce Hydra Branch, an auxiliary decoder without self-attention layers, to further enhance One-to-Many Relation Assignment by promoting different queries to make the same relation prediction. Hydra-SGG achieves state-of-the-art performance on multiple datasets, including VG150 (16.0 mR@50), Open Images V6 (50.1 weighted score), and GQA (12.7 mR@50).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。