仅用正样本对训练关键点检测与匹配,提升弱监督下的视觉几何任务性能。
RIPE++: Reinforced Keypoint Learning from Positive Pairs Only

- 基于正样本对设计几何一致性奖励,无需负样本对比
- 在MegaDepth1500上AUC@5提升至59.65,优于基线
- 适用于无相机位姿的低纹理医学视频,支持端到端弱监督
稀疏关键点提取与匹配是几何计算机视觉中结构重建、视觉SLAM、增强现实和医学图像配准等核心任务的基础。然而,学习鲁棒局部特征表示通常需要精确的相机位姿或深度监督,这些在真实场景中常不可得。强化学习(RL)作为替代方案,仅需判断两图是否为同一场景,但现有方法如RIPE依赖粗粒度二值奖励和精心构造的负样本对,限制了训练稳定性和描述子区分能力。本文重新审视基于强化学习的关键点学习,提出一种完全利用几何一致性信号的奖励机制,仅从单一正样本对中同时导出奖励与惩罚,无需负样本对比。该更丰富的监督信号足以在仅正样本对下学习判别性检测器与描述子,实现极低监督条件下的表示学习。此外,我们证明该相同强化学习目标可扩展至匹配阶段,通过改进LightGlue,使MegaDepth1500的AUC@5从56.58提升至59.65,并实现具有部分视觉重叠图像对的全流水线弱监督训练。我们在多个基准上验证了该方法,结果与全监督方法相当。进一步实验表明,该方法可应用于无相机位姿的低纹理医学视频序列,传统SfM流程在此类数据上常失效。代码与数据已公开于https://github.com/fraunhoferhhi/RIPEpp。
原文摘要 · Abstract (English)
Sparse keypoint extraction and matching underpin core tasks in geometric computer vision, including structure-from-motion, visual SLAM, augmented reality, and medical image registration. Learning robust local feature representations, however, typically requires accurate camera poses or depth supervision, which are often unavailable in real-world settings. Reinforcement learning (RL) has recently emerged as a promising alternative, requiring only the information if two images show the same scene or not. However, existing RL formulations such as RIPE rely on coarse binary rewards and carefully constructed negative training pairs, limiting training stability and descriptor discriminability. In this paper, we revisit RL-based keypoint learning and propose a reward that fully exploits the geometric consistency signal, deriving both reward and penalty from a single positive pair without contrasting against negatives. This richer signal provides sufficient supervisory contrast to learn discriminative detectors and descriptors from positive image pairs alone, enabling representation learning under extremely limited supervision. Furthermore, we show that the same RL objective can be extended to the matching stage by adapting LightGlue, raising AUC@5 on MegaDepth1500 from 56.58 to 59.65 and enabling weakly-supervised training of the full sparse matching pipeline from image pairs with partial visual overlap. We validate our approach on established benchmarks, demonstrating competitive results compared to fully-supervised methods. We further show that the method can be even trained on low texture medical video sequences, where camera poses are usually unavailable and standard SfM pipelines often fail. Code and data are available at https://github.com/fraunhoferhhi/RIPEpp .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。