用自编码器预训练天文图像，提升红移预测精度
AstroMAE: Redshift Prediction Using a Masked Autoencoder with a Novel Fine-Tuning Architecture
- 用遮蔽自编码器在SDSS图像上无监督预训练视觉变换器
- 在红移预测任务中超越多种视觉变换器与CNN模型
- 适合天文学数据挖掘与无监督学习研究者使用
红移预测是天文学中的基础任务，对理解宇宙膨胀和测定天体距离至关重要。机器学习方法凭借其高精度和快速性，为这一复杂任务提供了有前景的解决方案。然而，传统机器学习算法严重依赖标注数据和特定任务的特征提取。为此，我们提出AstroMAE，一种创新方法：在斯隆数字巡天（SDSS）图像上，利用遮蔽自编码器对视觉变换器编码器进行预训练。该方法使编码器无需标签即可捕捉数据中的全局模式。据我们所知，AstroMAE是首个将遮蔽自编码器应用于天文学数据的方法。预训练阶段忽略标签，使编码器获得数据的通用理解。随后，该预训练编码器在专为红移预测设计的精细调优架构中进行微调。我们在多种视觉变换器架构和基于CNN的模型上评估本模型，结果表明，AstroMAE的预训练模型和微调架构均表现更优。
原文摘要 · Abstract (English)
Redshift prediction is a fundamental task in astronomy, essential for understanding the expansion of the universe and determining the distances of astronomical objects. Accurate redshift prediction plays a crucial role in advancing our knowledge of the cosmos. Machine learning (ML) methods, renowned for their precision and speed, offer promising solutions for this complex task. However, traditional ML algorithms heavily depend on labeled data and task-specific feature extraction. To overcome these limitations, we introduce AstroMAE, an innovative approach that pretrains a vision transformer encoder using a masked autoencoder method on Sloan Digital Sky Survey (SDSS) images. This technique enables the encoder to capture the global patterns within the data without relying on labels. To the best of our knowledge, AstroMAE represents the first application of a masked autoencoder to astronomical data. By ignoring labels during the pretraining phase, the encoder gathers a general understanding of the data. The pretrained encoder is subsequently fine-tuned within a specialized architecture tailored for redshift prediction. We evaluate our model against various vision transformer architectures and CNN-based models, demonstrating the superior performance of AstroMAEs pretrained model and fine-tuning architecture.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。