arXiv:2512.11982astro-ph.IMcs.AI2025-12被引 3

用AI自动生成星系图像描述,实现超百万张星图的语义搜索。

Semantic search for 100M+ galaxy images using AI-generated captions

论文配图:Semantic search for 100M+ galaxy images using AI-generated captions
图 1 · 摘自论文原文
  • 用视觉语言模型生成星系图像描述,构建可搜索嵌入。
  • 在无特殊筛选数据下仍超越图像相似性搜索,发现36个新星流候选体。
  • 适合天文学家快速发现罕见天文现象,也适用于其他科学图像库。

通过手动标注发现科学现象效率低下,严重限制了对望远镜产生的数十亿张星系图像的探索。本文提出一种无需标签数据的语义搜索管道:利用视觉-语言模型(VLM)为星系图像生成描述,再将预训练天文学基础模型与这些描述嵌入进行对比对齐,从而大规模生成可搜索嵌入。实验表明,当前VLM生成的描述已足够用于训练出优于直接图像相似性搜索的语义模型。所提出的AION-Search模型在零样本条件下实现了领先性能,即使训练数据随机选取且未刻意包含罕见案例。此外,引入基于VLM的重排序方法,使最挑战目标在前100结果中的召回率几乎翻倍。首次实现对超过1亿张星系图像的灵活语义搜索,成功识别出36个新的银河系外恒星流候选体。本工作为大规模无标签科学图像档案提供语义可搜方案,拓展了从地球观测到显微成像等领域的数据探索能力。代码、数据与应用已公开于https://github.com/NolanKoblischke/AION-Search。

原文摘要 · Abstract (English)

Finding scientifically interesting phenomena through slow manual labeling campaigns severely limits our ability to explore the billions of galaxy images produced by telescopes. In this work, we develop a pipeline to create a semantic search engine from completely unlabeled image data. Our method leverages Vision-Language Models (VLMs) to generate descriptions for galaxy images, then contrastively aligns a pre-trained astronomy foundation model with these embedded descriptions to produce searchable embeddings at scale. We find that current VLMs provide descriptions that are sufficiently informative to train a semantic search model that outperforms direct image similarity search. Our model, AION-Search, achieves state-of-the-art zero-shot performance on finding rare phenomena despite training on randomly selected images with no deliberate curation for rare cases. Furthermore, we introduce a VLM-based re-ranking method that nearly doubles the recall for our most challenging targets in the top-100 results. For the first time, AION-Search enables flexible semantic search for over 100 million galaxy images, enabling discovery from previously infeasible searches, including the identification of 36 new extragalactic stellar stream candidates. More broadly, our work provides an approach for making large, unlabeled scientific image archives semantically searchable, expanding data exploration capabilities in fields from Earth observation to microscopy. The code, data, and app are publicly available at https://github.com/NolanKoblischke/AION-Search

星系搜索视觉语言模型语义搜索天文学

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。