arXiv:2412.11161cs.CV2024-12被引 4

通过知识引导构建稳定高效的匹配网络,显著提升跨谱图像块匹配性能。

Why and How: Knowledge-Guided Learning for Cross-Spectral Image Patch Matching

  • 基于特征一致性构建描述符与度量学习的桥梁,实现双向引导。
  • 引入硬负样本挖掘策略,使度量分支特征映射能力更强,性能大幅领先。
  • 无需复杂结构,在三个场景下均达到当前最优结果,适合跨模态匹配研究者。

基于特征关系学习的跨谱图像块匹配近年来受到广泛关注,但现有方法逐渐暴露出性能瓶颈。为解决此问题,本文首次探索描述符学习与度量学习之间的稳定高效桥梁,提出知识引导学习网络(KGL-Net),在不依赖复杂结构的前提下实现显著性能提升。具体而言,发现基于特征差异学习的度量学习与基于欧氏距离的描述符学习之间存在特征提取一致性,为构建桥梁提供基础。为保障桥梁稳定性与效率,一方面深入探索20种组合网络架构,另一方面设计特征引导损失函数实现特征互导。此外,不同于以往方法,本文强调度量分支的特征映射能力应受重视,因此提出针对度量学习的硬负样本挖掘策略(HNSM-M)。据我们所知,这是首次在度量网络中实现硬负样本挖掘,带来显著性能增益。大量实验表明,KGL-Net在三种不同跨谱图像块匹配场景中均达到当前最优(SOTA)性能。代码已开源:https://github.com/YuChuang1205/KGL-Net。

原文摘要 · Abstract (English)

Recently, cross-spectral image patch matching based on feature relation learning has attracted extensive attention. However, performance bottleneck problems have gradually emerged in existing methods. To address this challenge, we make the first attempt to explore a stable and efficient bridge between descriptor learning and metric learning, and construct a knowledge-guided learning network (KGL-Net), which achieves amazing performance improvements while abandoning complex network structures. Specifically, we find that there is feature extraction consistency between metric learning based on feature difference learning and descriptor learning based on Euclidean distance. This provides the foundation for bridge building. To ensure the stability and efficiency of the constructed bridge, on the one hand, we conduct an in-depth exploration of 20 combined network architectures. On the other hand, a feature-guided loss is constructed to achieve mutual guidance of features. In addition, unlike existing methods, we consider that the feature mapping ability of the metric branch should receive more attention. Therefore, a hard negative sample mining for metric learning (HNSM-M) strategy is constructed. To the best of our knowledge, this is the first time that hard negative sample mining for metric networks has been implemented and brings significant performance gains. Extensive experimental results show that our KGL-Net achieves SOTA performance in three different cross-spectral image patch matching scenarios. Our code are available at https://github.com/YuChuang1205/KGL-Net.

跨谱匹配特征学习度量学习硬负样本

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。