无需特征描述符的定位新方法,显著提升精度并支持多检测器训练。
GeoMix: Descriptor-Free Visual Localization via Global Context and Multi-Detector Training

- 通过方向与距离感知嵌入增强局部几何结构,提升匹配精度。
- 引入可学习上下文节点,用跨注意力聚合全局场景信息解决歧义。
- 支持多检测器联合训练,零样本泛化到未见检测器,逼近有描述符方法性能。
无描述符视觉定位可避免高维描述符存储,保护场景隐私并简化地图维护,但精度仍远低于基于描述符的方法。我们发现其瓶颈在于仅依赖几何信息时判别能力不足:当前方法未能充分利用局部几何线索,缺乏关键点间的全局上下文,且过度依赖单一检测器。进一步观察到,无描述符匹配天然支持多检测器训练,因异构关键点可在共享几何空间中优化,无需对齐描述符空间。基于此,提出GeoMix框架,在三个层面增强几何判别性:局部采用方向与距离感知嵌入,丰富邻域聚合的空间结构;全局引入可学习上下文节点,通过跨注意力聚合并重分配全场景信息,解决局部感受野外的歧义;训练层面采用混合训练(Mix-Training),在检测器无关的几何空间中联合学习多个检测器的表示。在MegaDepth、Cambridge Landmarks、7Scenes和Aachen Day-Night数据集上的大量实验表明,GeoMix在无描述符方法中达到新基准,75百分位旋转误差降低89%,平移误差最高降低90%,且零样本泛化至未见检测器,缩小与基于描述符方法的差距。代码已公开于https://github.com/YejunZhang/Geomix。
原文摘要 · Abstract (English)
Descriptor-free visual localization eliminates high-dimensional descriptor storage, preserves scene privacy, and simplifies map maintenance, yet its accuracy still lags far behind descriptor-based pipelines. We identify this gap to insufficient geometric discriminability in geometry-only matching. Without visual appearance, current methods underutilize local geometry cues, lack the global context among keypoints, and overfit to a single keypoint detector. We further observe that descriptor-free matching naturally enables multi-detector training, as heterogeneous keypoints can be optimized in a shared geometry-only space without aligning descriptor spaces. Building on these insights, we propose GeoMix, a descriptor-free 2D-3D matching framework that strengthens geometric discriminability at three levels. Locally, directional and distance-aware embeddings enrich neighborhood aggregation with fine-grained spatial structure. Globally, learnable context nodes aggregate and redistribute scene-wide information via cross-attention to resolve ambiguities beyond local receptive fields. At the training level, Mix-Training exploits this detector-agnostic geometry space to learn representations across multiple keypoint detectors. Extensive experiments on MegaDepth, Cambridge Landmarks, 7Scenes, and Aachen Day-Night show that GeoMix sets a new state of the art among descriptor-free methods, reducing 75th-percentile rotation error by 89\% and translation error by up to 90\% over the previous best, while generalizing zero-shot to unseen detectors and narrowing the gap to descriptor-based pipelines. Code is available at $\href{https://github.com/YejunZhang/Geomix}{\text{this links}}$.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。