让视觉特征同时懂语义和几何，推理快98%以上
GECO: Geometrically Consistent Embedding with Lightspeed Inference
- 用最优传输框架实现几何一致的特征学习
- 在三个数据集上准确率提升4.1%~6.2%，达当前最佳
- 适合需要精确空间关系理解的任务，如3D重建
近期视觉特征学习进展表明，自监督视觉基础模型可捕捉语义对应关系，但常缺乏对底层3D几何的认知。GECO通过生成几何一致的特征，使特征能基于几何区分物体部分（如左右眼、前后腿）。我们提出一种基于最优传输的训练框架，可在关键点被遮挡或消失时仍提供监督信号。采用轻量级架构，GECO 推理速度达30帧/秒，比之前方法快98.2%，在PFPascal、APK和CUB数据集上分别将PCK指标提升6.0%、6.2%和4.1%，达到当前最优表现。最后，我们指出仅用PCK无法衡量几何质量，提出了新评估指标与洞察，推动更几何感知的特征学习。项目页面：https://reginehartwig.github.io/publications/geco/
原文摘要 · Abstract (English)
Recent advances in feature learning have shown that self-supervised vision foundation models can capture semantic correspondences but often lack awareness of underlying 3D geometry. GECO addresses this gap by producing geometrically coherent features that semantically distinguish parts based on geometry (e.g., left/right eyes, front/back legs). We propose a training framework based on optimal transport, enabling supervision beyond keypoints, even under occlusions and disocclusions. With a lightweight architecture, GECO runs at 30 fps, 98.2% faster than prior methods, while achieving state-of-the-art performance on PFPascal, APK, and CUB, improving PCK by 6.0%, 6.2%, and 4.1%, respectively. Finally, we show that PCK alone is insufficient to capture geometric quality and introduce new metrics and insights for more geometry-aware feature learning. Link to project page: https://reginehartwig.github.io/publications/geco/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。