用强化学习提升关键点在序列中的追踪质量，解决光照变化下的匹配难题。
From Pairs to Sequences: Track-Aware Policy Gradients for Keypoint Detection
- 将关键点检测建模为序列决策问题，引入跟踪感知的奖励机制。
- 在相对位姿估计和三维重建任务中显著优于现有SOTA方法。
- 适合需要长序列稳定匹配的3D视觉系统开发者使用。
基于关键点的匹配是现代3D视觉系统（如运动结构重建SfM和即时定位与地图构建SLAM）的核心组件。现有多数学习方法在图像对上训练，无法显式优化关键点在视角与光照剧烈变化下跨序列的长期可追踪性。本文将关键点检测重新建模为序列决策问题，提出TraqPoint——一种端到端的强化学习框架，直接优化关键点的跟踪质量（Traq）。其核心创新在于设计了一种跟踪感知的奖励机制，联合鼓励关键点在多视图间保持一致性和区分性，通过策略梯度方法实现优化。在稀疏匹配基准测试（包括相对位姿估计和3D重建）上的大量实验表明，TraqPoint显著优于多个当前最优（SOTA）的关键点检测与描述方法。
原文摘要 · Abstract (English)
Keypoint-based matching is a fundamental component of modern 3D vision systems, such as Structure-from-Motion (SfM) and SLAM. Most existing learning-based methods are trained on image pairs, a paradigm that fails to explicitly optimize for the long-term trackability of keypoints across sequences under challenging viewpoint and illumination changes. In this paper, we reframe keypoint detection as a sequential decision-making problem. We introduce TraqPoint, a novel, end-to-end Reinforcement Learning (RL) framework designed to optimize the \textbf{Tra}ck-\textbf{q}uality (Traq) of keypoints directly on image sequences. Our core innovation is a track-aware reward mechanism that jointly encourages the consistency and distinctiveness of keypoints across multiple views, guided by a policy gradient method. Extensive evaluations on sparse matching benchmarks, including relative pose estimation and 3D reconstruction, demonstrate that TraqPoint significantly outperforms some state-of-the-art (SOTA) keypoint detection and description methods.The code will be available at https://github.com/xiaomi-research/traqpoint.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。