ReRAW高效还原RGB图像为原始感光数据，提升边缘设备检测性能
ReRAW: RGB-to-RAW Image Reconstruction via Stratified Sampling for Efficient Object Detection on the Edge
- 多头结构在伽马空间预测原始图像候选
- 分层采样策略优化亮像素重建效果
- 合成与真实原始数据联合预训练更优
基于边缘的计算机视觉模型在资源受限设备上运行时，使用未处理的、细节丰富的原始图像（RAW）数据比经过处理的RGB图像更具优势。然而，训练这些模型需要大规模标注的RAW数据集，而这类数据获取成本高且不切实际。因此，将现有标注的RGB数据集转换为特定传感器的原始图像变得至关重要。本文提出ReRAW，一种实现跨五个不同原始数据集最先进的图像重建性能的RGB到原始图像转换模型。其创新之处在于采用多头架构，在伽马空间中预测原始图像候选，并通过基于分层采样的训练数据选择策略，进一步提升对较亮原始像素的重建效果。最终实验表明，使用由ReRAW生成的高质量合成原始数据和真实原始图像联合预训练紧凑模型，在下游目标检测任务中表现优于标准RGB流水线，也优于仅用RGB预训练模型进行原始图像微调的方案。
原文摘要 · Abstract (English)
Edge-based computer vision models running on compact, resource-limited devices benefit greatly from using unprocessed, detail-rich RAW sensor data instead of processed RGB images. Training these models, however, necessitates large labeled RAW datasets, which are costly and often impractical to obtain. Thus, converting existing labeled RGB datasets into sensor-specific RAW images becomes crucial for effective model training. In this paper, we introduce ReRAW, an RGB-to-RAW conversion model that achieves state-of-the-art reconstruction performance across five diverse RAW datasets. This is accomplished through ReRAW's novel multi-head architecture predicting RAW image candidates in gamma space. The performance is further boosted by a stratified sampling-based training data selection heuristic, which helps the model better reconstruct brighter RAW pixels. We finally demonstrate that pretraining compact models on a combination of high-quality synthetic RAW datasets (such as generated by ReRAW) and ground-truth RAW images for downstream tasks like object detection, outperforms both standard RGB pipelines, and RAW fine-tuning of RGB-pretrained models for the same task.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。