arXiv:2409.18049cs.CVcs.AI2024-09ECCV被引 37

用图像片段替代整图,提升重访场景识别准确率。

Revisit Anything: Visual Place Recognition via Image Segment Retrieval

论文配图:Revisit Anything: Visual Place Recognition via Image Segment Retrieval
图 1 · 摘自论文原文
  • 将图像分解为语义片段,构建带邻接关系的SuperSegment表示
  • 在多个基准数据集上实现最新识别召回率,优于传统整图方法
  • 适用于通用与专用编码器,可拓展至物体目标导航任务

准确识别重访场景对具身智能体定位与导航至关重要,但相机视角和场景外观的强烈变化使视觉表征难以区分。现有方法编码整图并搜索匹配,但在不同视角下相同地点的图像中,重叠部分的相似性易被非重叠部分的差异掩盖。为此,本文提出基于图像片段的识别方法:利用开放集图像分割将图像分解为有意义的实体(物体与背景),构建包含局部邻接关系的SuperSegment表示;设计因子化特征聚合机制,高效生成紧凑向量表示。实验表明,检索部分片段表示显著提升识别召回率。所提方法名为SegVLAD,在多个基准数据集上达到新最优性能,且兼容通用与任务专用图像编码器。此外,该方法还成功应用于物体实例检索任务,打通视觉场景识别与物体目标导航的研究鸿沟,实现‘重访任何事物’。代码已开源。

原文摘要 · Abstract (English)

Accurately recognizing a revisited place is crucial for embodied agents to localize and navigate. This requires visual representations to be distinct, despite strong variations in camera viewpoint and scene appearance. Existing visual place recognition pipelines encode the "whole" image and search for matches. This poses a fundamental challenge in matching two images of the same place captured from different camera viewpoints: "the similarity of what overlaps can be dominated by the dissimilarity of what does not overlap". We address this by encoding and searching for "image segments" instead of the whole images. We propose to use open-set image segmentation to decompose an image into `meaningful' entities (i.e., things and stuff). This enables us to create a novel image representation as a collection of multiple overlapping subgraphs connecting a segment with its neighboring segments, dubbed SuperSegment. Furthermore, to efficiently encode these SuperSegments into compact vector representations, we propose a novel factorized representation of feature aggregation. We show that retrieving these partial representations leads to significantly higher recognition recall than the typical whole image based retrieval. Our segments-based approach, dubbed SegVLAD, sets a new state-of-the-art in place recognition on a diverse selection of benchmark datasets, while being applicable to both generic and task-specialized image encoders. Finally, we demonstrate the potential of our method to ``revisit anything'' by evaluating our method on an object instance retrieval task, which bridges the two disparate areas of research: visual place recognition and object-goal navigation, through their common aim of recognizing goal objects specific to a place. Source code: https://github.com/AnyLoc/Revisit-Anything.

场景识别图像分割具身智能检索

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。