用合成数据+少量真实标注,提升田间杂草环境下的植物分割精度。
Enabling Plant Phenotyping in Weedy Environments using Multi-Modal Imagery via Synthetic and Generated Training Data
- 用1128张合成图像训练模型,生成作物与杂草的分割掩码。
- 仅用5张真实标注图,杂草分割准确率提升22%,作物提升17%。
- 通过CycleGAN-turbo实现RGB到热成像的跨模态对齐,无需标定。
在户外环境下,由于作物与杂草对比度低且频繁遮挡,热成像中的精准植物分割仍是高通量田间表型分析的重大挑战。为此,我们提出一种框架,利用合成RGB图像、少量真实标注以及基于GAN的跨模态对齐,增强热成像的语义分割性能。模型在包含复杂作物与杂草混合的1,128张合成图像上训练,生成作物和杂草的分割掩码。此外,评估了在训练中加入仅5张人工标注的真实田间图像的效果,采用不同采样策略。当将全部合成数据与少量标注的真实图像结合时,相比全真实数据基线,杂草类别的相对准确率提升22%,作物类别提升17%。通过CycleGAN-turbo将RGB图像转换为热成像,实现鲁棒的模板匹配而无需校准。结果表明,结合合成数据、有限的人工标注及生成模型的跨域转换,可显著提升复杂田间环境中多模态图像的分割表现。
原文摘要 · Abstract (English)
Accurate plant segmentation in thermal imagery remains a significant challenge for high throughput field phenotyping, particularly in outdoor environments where low contrast between plants and weeds and frequent occlusions hinder performance. To address this, we present a framework that leverages synthetic RGB imagery, a limited set of real annotations, and GAN-based cross-modality alignment to enhance semantic segmentation in thermal images. We trained models on 1,128 synthetic images containing complex mixtures of crop and weed plants in order to generate image segmentation masks for crop and weed plants. We additionally evaluated the benefit of integrating as few as five real, manually segmented field images within the training process using various sampling strategies. When combining all the synthetic images with a few labeled real images, we observed a maximum relative improvement of 22% for the weed class and 17% for the plant class compared to the full real-data baseline. Cross-modal alignment was enabled by translating RGB to thermal using CycleGAN-turbo, allowing robust template matching without calibration. Results demonstrated that combining synthetic data with limited manual annotations and cross-domain translation via generative models can significantly boost segmentation performance in complex field environments for multi-model imagery.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。