统一视频场景图生成，用同一模型处理物体和关系的粗粒度与细粒度任务
UNO: Unifying One-stage Video Scene Graph Generation via Object-Centric Visual Representation Learning
- 基于对象中心的槽注意力机制，分解视觉特征为物体与关系槽
- 在两个基准上均达先进水平，且比多阶段方法更高效
- 适合需要统一视频理解框架的研究者和开发者
视频场景图生成（VidSGG）旨在通过检测物体并建模其随时间演化的交互关系，以结构化图的形式表征动态视觉内容。以往研究通常聚焦于粗粒度的框级或细粒度的全景像素级生成，常需特定任务架构和多阶段训练流程。本文提出UNO（UNified Object-centric VidSGG），一个单阶段、统一的端到端框架，可同时处理两类任务。其核心是扩展的槽注意力机制，将视觉特征分解为物体和关系槽；为增强时序建模，引入物体时序一致性学习，确保跨帧表示一致且无需显式追踪模块；另设动态三元组预测模块，将关系槽与对应物体对关联，捕捉动态交互。在标准框级与像素级基准上评估显示，UNO不仅在两项任务中均达到先进性能，且凭借统一的对象中心设计提升了效率。代码已开源。
原文摘要 · Abstract (English)
Video Scene Graph Generation (VidSGG) aims to represent dynamic visual content by detecting objects and modeling their temporal interactions as structured graphs. Prior studies typically target either coarse-grained box-level or fine-grained panoptic pixel-level VidSGG, often requiring task-specific architectures and multi-stage training pipelines. In this paper, we present UNO (UNified Object-centric VidSGG), a single-stage, unified framework that jointly addresses both tasks within an end-to-end architecture. UNO is designed to minimize task-specific modifications and maximize parameter sharing, enabling generalization across different levels of visual granularity. The core of UNO is an extended slot attention mechanism that decomposes visual features into object and relation slots. To ensure robust temporal modeling, we introduce object temporal consistency learning, which enforces consistent object representations across frames without relying on explicit tracking modules. Additionally, a dynamic triplet prediction module links relation slots to corresponding object pairs, capturing evolving interactions over time. We evaluate UNO on standard box-level and pixel-level VidSGG benchmarks. Results demonstrate that UNO not only achieves competitive performance across both tasks but also offers improved efficiency through a unified, object-centric design. Code is available at: https://github.com/Fsoft-AIC/UNO
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。