通过训练光子相关性,提升极暗环境下物体识别准确率
Ultra-low-light computer vision using trained photon correlations
- 联合训练光子相关光源与Transformer模型,让算法利用光子空间相关性
- 在极低光照下分类准确率最高提升15个百分点,优于传统方法
- 适合需在极低光条件下做目标识别的科研与工程应用
使用关联光子源照明已被证实可通过利用信号光子的空间相关性(而噪声点击不相关)来实现从嘈杂相机帧中高保真重建图像。然而,在计算机视觉任务中,目标通常并非最终重建图像,而是对场景进行推断(如识别物体)。本文展示如何将关联光子照明用于混合光学-电子计算机视觉流水线以实现物体识别优势。我们提出相关性感知训练(CAT):在少量(≤100次)采样下,端到端优化可训练的关联光子照明源与Transformer后端,使后者学会利用光子相关性。在超低光和噪声成像条件下,分类准确率相比传统非关联照明方法最高提升15个百分点,且优于未训练的关联照明方案。本工作表明,针对物体识别任务专门设计,并联合训练光子相关模式与数字后端,可在极低光子预算场景下突破现有以图像重建为目标的方法极限。
原文摘要 · Abstract (English)
Illumination using correlated photon sources has been established as an approach to allowing high-fidelity images to be reconstructed from noisy camera frames by taking advantage of the knowledge that signal photons are spatially correlated whereas detector clicks due to noise are uncorrelated. However, in computer-vision tasks, the goal is often not ultimately to reconstruct an image, but to make inferences about a scene -- such as what object is present. Here we show how correlated-photon illumination can be used to gain an advantage in a hybrid optical-electronic computer-vision pipeline for object recognition. We demonstrate correlation-aware training (CAT): end-to-end optimization of a trainable correlated-photon illumination source and a Transformer backend in a way that the Transformer can learn to benefit from the correlations, using a small number (<= 100) of shots. We show a classification accuracy enhancement of up to 15 percentage points over conventional, uncorrelated-illumination-based computer vision in ultra-low-light and noisy imaging conditions, as well as an improvement over using untrained correlated-photon illumination. Our work illustrates how specializing to a computer-vision task -- object recognition -- and training the pattern of photon correlations in conjunction with a digital backend allows us to push the limits of accuracy in highly photon-budget-constrained scenarios beyond existing methods focused on image reconstruction.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。