用多视角对比学习提升自然图像分类能力，缺一视角仍有效。
Contrastive ground-level image and remote sensing pre-training improves representation learning for natural world imagery
- 通过地面与航拍图像对比预训练学习跨视角表征
- 在物种识别任务中实现94.3%准确率，缺一视角仍保持高精度
- 适合生态学、计算机视觉领域研究者参考
多模态图像-文本对比学习已证明可在不同模态间学习联合表征。本文展示：即使某一视角缺失，利用多视角图像数据进行对比学习也能提升下游细粒度分类性能。提出一种名为CRISP的新预训练任务，用于自然世界中地面与航空图像的表征学习，并构建了包含超过300万对图像的自然世界多视角数据集NMV，覆盖加利福尼亚州6,000余种植物类群。该数据集及相关材料已公开于hf.co/datasets/andyvhuynh/NatureMultiView。
原文摘要 · Abstract (English)
Multimodal image-text contrastive learning has shown that joint representations can be learned across modalities. Here, we show how leveraging multiple views of image data with contrastive learning can improve downstream fine-grained classification performance for species recognition, even when one view is absent. We propose ContRastive Image-remote Sensing Pre-training (CRISP)$\unicode{x2014}$a new pre-training task for ground-level and aerial image representation learning of the natural world$\unicode{x2014}$and introduce Nature Multi-View (NMV), a dataset of natural world imagery including $>3$ million ground-level and aerial image pairs for over 6,000 plant taxa across the ecologically diverse state of California. The NMV dataset and accompanying material are available at hf.co/datasets/andyvhuynh/NatureMultiView.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。