让模型学会看物体位置关系,提升自监督学习的泛化能力。
Learning to Perceive "Where": Spatial Pretext Tasks for Robust Self-Supervised Learning

- 设计空间预测任务,让模型预测图像局部区域的相对位置与尺度。
- 在图像识别、分割等任务上性能全面提升,尤其增强分布外鲁棒性。
- 适合关注视觉结构理解与模型泛化性的研究者使用。
现有自监督学习方法多关注物体不变特征,忽视了物体内部结构的空间关系。为此,本文提出空间预测(SP),一种基于连续几何空间的预训练回归任务,通过预测同一图像中两段解耦局部视图间的相对位置和尺度,建模部件间关系。该方法可作为独立模块无缝嵌入多种自监督框架。大量实验表明,其在图像识别、细粒度分类、语义分割和深度估计任务中均实现一致提升,并显著增强物体识别的分布外鲁棒性。为评估空间推理能力,引入两个新任务:图像块对的位置与尺度预测,以及拼图重构后的重新排序与识别任务。优异表现证明模型具备更强的空间结构感知能力。结果表明,显式建模空间信息是一种有效的归纳偏置,能生成更结构化的表征并提升泛化性能。代码与模型将公开。
原文摘要 · Abstract (English)
Existing self-supervised learning (SSL) methods primarily learn object-invariant representations but often neglect the spatial structure and relationships among object parts. To address this limitation, we introduce Spatial Prediction (SP), a spatially aware pretext regression task that predicts the relative position and scale between a pair of disentangled local views from the same image. By modeling part-to-part relationships in a continuous geometric space, SP encourages representations to capture fine-grained spatial dependencies beyond invariant categorical semantics, thereby learning the compositional structure of visual scenes. SP is implemented as a decoupled plug-in and can be seamlessly integrated into diverse SSL frameworks. Extensive experiments show consistent improvements across image recognition, fine-grained classification, semantic segmentation, and depth estimation, as well as substantial gains in out-of-distribution robustness for object recognition. To evaluate spatial reasoning, we introduce (1) a position and scale prediction task on image patch pairs and (2) a jigsaw understanding task requiring patch reordering and recognition after reconstruction. Strong performance on these tasks indicates improved spatial structure and geometric awareness. Overall, explicitly modeling spatial information provides an effective inductive bias for SSL, leading to more structured representations and better generalization. Code and models will be released.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。