多传感器融合提升农业场景下光照不良时的3D重建质量
AgriNeRF: Neural Radiance Fields for Agriculture in Challenging Lighting Conditions
- 融合RGB、事件与热成像相机，构建抗光照干扰的神经辐射场
- 水果检测准确率提升43%（mAP50）和61%（mAP50-95）
- 适合需要全天候精准果园监测的智能农业应用
神经辐射场（NeRF）在3D场景重建与新视角合成中展现出巨大潜力。在农业场景中，NeRF可作为数字孪生，为产量估算和果实检测提供关键信息。然而，传统NeRF对低光、强光及变化光照等挑战性条件不鲁棒。为此，本研究采用RGB相机、事件相机与热成像相机三种传感器。基于RGB的场景重建在PSNR上提升2.06 dB，SSIM提升8.3%；跨谱重建使下游果实检测的mAP50提升43.0%，mAP50-95提升61.1%。多模态融合显著增强了NeRF的鲁棒性与信息量。实验表明，该系统在不同树冠覆盖与时段下均能生成高质量逼真重建结果。本工作实现了在视觉退化场景下表现优异的鲁棒NeRF，以及用于自动果实检测的跨谱表征学习。
原文摘要 · Abstract (English)
Neural Radiance Fields (NeRFs) have shown significant promise in 3D scene reconstruction and novel view synthesis. In agricultural settings, NeRFs can serve as digital twins, providing critical information about fruit detection for yield estimation and other important metrics for farmers. However, traditional NeRFs are not robust to challenging lighting conditions, such as low-light, extreme bright light and varying lighting. To address these issues, this work leverages three different sensors: an RGB camera, an event camera and a thermal camera. Our RGB scene reconstruction shows an improvement in PSNR and SSIM by +2.06 dB and +8.3% respectively. Our cross-spectral scene reconstruction enhances downstream fruit detection by +43.0% in mAP50 and +61.1% increase in mAP50-95. The integration of additional sensors leads to a more robust and informative NeRF. We demonstrate that our multi-modal system yields high quality photo-realistic reconstructions under various tree canopy covers and at different times of the day. This work results in the development of a resilient NeRF, capable of performing well in visibly degraded scenarios, as well as a learnt cross-spectral representation, that is used for automated fruit detection.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。