仅用一张图片，一次推理就预测场景中所有物体的前后顺序。
Holistic Order Prediction in Natural Scenes
- 通过对象查询与掩码描述符的交互，实现全局顺序推理。
- 单次前向传播即可输出全部物体的遮挡与深度顺序。
- 无需类别标签或分割掩码，适合实际应用部署。
即使在受控环境下，理解实例级几何结构对多种视觉模型仍是挑战。现有专门系统依赖昂贵输入（类别标签、二值分割掩码）和高计算成本（二次数量级前向传播）。我们提出 InstaFormer，一种能够进行整体顺序预测的网络：仅需输入一张RGB图像，即可在一次前向传播中输出场景中所有实例的完整遮挡与深度顺序。其核心机制在于对象查询与语义表示同一对象但携带互补信息的潜在掩码描述符之间的交互。我们全面评估并消融了该方法的有效性。代码与模型已开源，地址为 https://github.com/SNU-VGILab/InstaOrder。
原文摘要 · Abstract (English)
Even in controlled settings, understanding instance-wise geometries is a challenging task for a wide range of visual models. Although specialized systems exist, modern arts rely on expensive input formats (category labels, binary segmentation masks) and inference costs (a quadratic amount of forward passes). We mitigate these limitations by proposing InstaFormer, a network capable of holistic order prediction. That is, solely given an input RGB image, InstaFormer returns the full occlusion and depth orderings for all the instances in the scene in a single forward pass. At its core, InstaFormer relies on interactions between object queries and latent mask descriptors that semantically represent the same objects while carrying complementary information. We comprehensively benchmark and ablate our approach to highlight its effectiveness. Our code and models are open-source and available at this URL: https://github.com/SNU-VGILab/InstaOrder.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。