REACT++ 实现了实时场景图生成的高效与高精度平衡
REACT++: Efficient Cross-Attention for Real-Time Scene Graph Generation
- 通过原型空间中的跨注意力机制优化特征提取效率
- 推理速度最快，关系预测准确率提升10%，物体检测不降级
- 适合需要快速生成视觉关系图的应用场景
场景图生成（SGG）将图像中物体间的视觉关系编码为图结构，是具身智能体推理等下游任务的基础。为支持实时应用，SGG需在性能与推理速度间取得平衡。现有方法多侧重于提升关系预测或物体检测准确率，或降低延迟，难以兼顾三者。本文基于先前的REACT架构，提出REACT++，通过原型空间中的高效特征提取与主体-对象跨注意力机制，在保持物体检测性能的同时，显著提升关系预测准确率，并实现现有模型中最快速度。相比前代REACT，REACT++平均提速20%，关系预测准确率提升10%。代码已开源：https://github.com/Maelic/SGG-Benchmark。
原文摘要 · Abstract (English)
Scene Graph Generation (SGG) is a task that encodes visual relationships between objects in images as graph structures. SGG shows significant promise as a foundational component for downstream tasks, such as reasoning for embodied agents. To enable real-time applications, SGG must address the trade-off between performance and inference speed. However, current methods tend to focus on one of the following: (1) improving relation prediction accuracy, (2) enhancing object detection accuracy, or (3) reducing latency, without aiming to balance all three objectives simultaneously. To address this limitation, we build on the powerful Real-time Efficiency and Accuracy Compromise for Tradeoffs in Scene Graph Generation (REACT) architecture and propose REACT++, a new state-of-the-art model for real-time SGG. By leveraging efficient feature extraction and subject-to-object cross-attention within the prototype space, REACT++ balances latency and representational power. REACT++ achieves the highest inference speed among existing SGG models, improving relation prediction accuracy without sacrificing object detection performance. Compared to the previous REACT version, REACT++ is 20% faster with a gain of 10% in relation prediction accuracy on average. The code is available at https://github.com/Maelic/SGG-Benchmark.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。