仅用图像对是否相同标签,训练出鲁棒关键点提取器。
RIPE: Reinforcement Learning on Unlabeled Image Pairs for Robust Keypoint Extraction
- 用强化学习+二值标签训练关键点检测与描述
- 在多个基准上达到顶尖性能,无需人工变换或3D数据
- 适合无标注数据场景下的关键点模型开发
我们提出RIPE,一种基于强化学习的弱监督框架,用于训练关键点提取器,在检测与描述任务中表现优异。不同于依赖人工变换、预生成模型或3D数据的传统方法,RIPE仅需判断图像对是否属于同一场景的二值标签,极大拓展了可用训练数据规模,从而构建出高度泛化且鲁棒的关键点提取器。RIPE采用超列方法,利用编码器中间层特征融合多尺度信息以描述关键点,并引入辅助损失提升描述子的区分能力。在标准基准上的全面评估表明,RIPE简化了数据准备流程,性能媲美当前最优方法,标志着关键点提取与描述的重要进展。代码已开源:https://github.com/fraunhoferhhi/RIPE。
原文摘要 · Abstract (English)
We introduce RIPE, an innovative reinforcement learning-based framework for weakly-supervised training of a keypoint extractor that excels in both detection and description tasks. In contrast to conventional training regimes that depend heavily on artificial transformations, pre-generated models, or 3D data, RIPE requires only a binary label indicating whether paired images represent the same scene. This minimal supervision significantly expands the pool of training data, enabling the creation of a highly generalized and robust keypoint extractor. RIPE utilizes the encoder's intermediate layers for the description of the keypoints with a hyper-column approach to integrate information from different scales. Additionally, we propose an auxiliary loss to enhance the discriminative capability of the learned descriptors. Comprehensive evaluations on standard benchmarks demonstrate that RIPE simplifies data preparation while achieving competitive performance compared to state-of-the-art techniques, marking a significant advancement in robust keypoint extraction and description. To support further research, we have made our code publicly available at https://github.com/fraunhoferhhi/RIPE.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。