用强化学习训练无需描述子的多样关键点检测器，提升三维重建效果。
DaD: Distilled Reinforcement Learning for Diverse Keypoint Detection
- 通过强化学习设计无描述子自监督目标，避免依赖外部特征。
- 提出平衡top-K采样防止训练崩溃，实现可竞争性能。
- 融合明暗检测器差异，提升关键点多样性与泛化能力。
关键点使结构光流（SfM）系统能够扩展到数千张图像。然而，设计关键点检测目标颇具挑战，因为SfM不可微。通常依赖描述子的辅助目标进行优化，但这引入了对描述子的依赖，不理想。本文提出一种完全自监督且无需描述子的关键点检测目标，基于强化学习实现。为防止训练退化，采用平衡的top-K采样策略。尽管已有良好表现，但发现两类检测器分别仅能检测亮或暗关键点。为此，我们训练第三种检测器DaD，通过最小化明暗检测器点最大值的KL散度来优化。该方法在多个基准上显著超越现有最佳水平。代码与模型权重已在https://github.com/parskatt/dad公开。
原文摘要 · Abstract (English)
Keypoints are what enable Structure-from-Motion (SfM) systems to scale to thousands of images. However, designing a keypoint detection objective is a non-trivial task, as SfM is non-differentiable. Typically, an auxiliary objective involving a descriptor is optimized. This however induces a dependency on the descriptor, which is undesirable. In this paper we propose a fully self-supervised and descriptor-free objective for keypoint detection, through reinforcement learning. To ensure training does not degenerate, we leverage a balanced top-K sampling strategy. While this already produces competitive models, we find that two qualitatively different types of detectors emerge, which are only able to detect light and dark keypoints respectively. To remedy this, we train a third detector, DaD, that optimizes the Kullback-Leibler divergence of the pointwise maximum of both light and dark detectors. Our approach significantly improve upon SotA across a range of benchmarks. Code and model weights are publicly available at https://github.com/parskatt/dad
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。