用自监督学习融合近红外与中红外光谱,提升土壤属性预测精度。
Self-supervised and Multi-fidelity Learning for Extended Predictive Soil Spectroscopy
- 基于自编码器构建光谱嵌入的潜在空间,仅用无标签数据预训练。
- 将近红外转中红外的转换结果在9个土壤属性上表现优于纯近红外模型。
- 适合需要低成本设备但又想利用高质量中红外数据的研究者。
我们提出一种自监督机器学习(SSML)框架,用于多保真度学习和扩展的土壤光谱预测。通过大型中红外(MIR)光谱库和变分自编码器算法,对未标记的光谱数据进行预训练,获得压缩的潜在空间以生成光谱嵌入。在此阶段,仅使用无标签数据,从而充分利用完整的光谱数据库及扫描重复样本进行增强训练。同时,冻结训练好的MIR解码器,将其接入近红外(NIR)编码器,以低代价实现从NIR到MIR的光谱转换,从而利用大容量的MIR库中蕴含的预测能力。该任务基于少量带有配对光谱的KSSL库完成。随后,下游机器学习模型被训练,用于在原始光谱、预测光谱与潜在空间嵌入之间建立映射,以预测九种土壤属性。性能在独立于KSSL训练数据的金标准测试集上评估,并采用回归拟合优度指标。相比基线模型,所提的SSML及其嵌入在所有土壤属性预测任务中均达到相似或更优的准确率。由光谱转换(NIR→MIR)产生的预测虽未达到原始MIR光谱水平,但优于纯NIR模型,表明统一的光谱潜在空间能有效利用更大更丰富的MIR数据集,提升当前NIR库中代表性不足的土壤属性预测效果。
原文摘要 · Abstract (English)
We propose a self-supervised machine learning (SSML) framework for multi-fidelity learning and extended predictive soil spectroscopy based on latent space embeddings. A self-supervised representation was pretrained with the large MIR spectral library and the Variational Autoencoder algorithm to obtain a compressed latent space for generating spectral embeddings. At this stage, only unlabeled spectral data were used, allowing us to leverage the full spectral database and the availability of scan repeats for augmented training. We also leveraged and froze the trained MIR decoder for a spectrum conversion task by plugging it into a NIR encoder to learn the mapping between NIR and MIR spectra in an attempt to leverage the predictive capabilities contained in the large MIR library with a low cost portable NIR scanner. This was achieved by using a smaller subset of the KSSL library with paired NIR and MIR spectra. Downstream machine learning models were then trained to map between original spectra, predicted spectra, and latent space embeddings for nine soil properties. The performance of was evaluated independently of the KSSL training data using a gold-standard test set, along with regression goodness-of-fit metrics. Compared to baseline models, the proposed SSML and its embeddings yielded similar or better accuracy in all soil properties prediction tasks. Predictions derived from the spectrum conversion (NIR to MIR) task did not match the performance of the original MIR spectra but were similar or superior to predictive performance of NIR-only models, suggesting the unified spectral latent space can effectively leverage the larger and more diverse MIR dataset for prediction of soil properties not well represented in current NIR libraries.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。