让图像信号处理自动适配检测任务,动态调节计算量。
AdaptiveISP: Learning an Adaptive Image Signal Processor for Object Detection
- 用强化学习自动设计可变处理流程的ISP
- 在复杂场景下检测准确率超越现有方法
- 适合动态范围大、需平衡性能与算力的场景
图像信号处理器(ISP)将原始传感器信号转换为数字图像,显著影响图像质量及下游视觉任务表现。现有方法多以提升图像质量为目标,但对检测等高层任务优化不足;且训练后ISP结构固定,难以应对动态场景。为此,我们提出AdaptiveISP,一种任务驱动且场景自适应的ISP。关键观察是:多数输入只需少量处理模块即可提升检测性能,仅少数输入需要复杂处理。基于此,AdaptiveISP利用深度强化学习,自动生成最优ISP结构和参数,以最大化检测性能。实验表明,AdaptiveISP不仅优于现有最先进方法,还能动态平衡检测精度与计算开销,尤其适用于动态范围变化大的场景。项目主页:https://openimaginglab.github.io/AdaptiveISP/
原文摘要 · Abstract (English)
Image Signal Processors (ISPs) convert raw sensor signals into digital images, which significantly influence the image quality and the performance of downstream computer vision tasks. Designing ISP pipeline and tuning ISP parameters are two key steps for building an imaging and vision system. To find optimal ISP configurations, recent works use deep neural networks as a proxy to search for ISP parameters or ISP pipelines. However, these methods are primarily designed to maximize the image quality, which are sub-optimal in the performance of high-level computer vision tasks such as detection, recognition, and tracking. Moreover, after training, the learned ISP pipelines are mostly fixed at the inference time, whose performance degrades in dynamic scenes. To jointly optimize ISP structures and parameters, we propose AdaptiveISP, a task-driven and scene-adaptive ISP. One key observation is that for the majority of input images, only a few processing modules are needed to improve the performance of downstream recognition tasks, and only a few inputs require more processing. Based on this, AdaptiveISP utilizes deep reinforcement learning to automatically generate an optimal ISP pipeline and the associated ISP parameters to maximize the detection performance. Experimental results show that AdaptiveISP not only surpasses the prior state-of-the-art methods for object detection but also dynamically manages the trade-off between detection performance and computational cost, especially suitable for scenes with large dynamic range variations. Project website: https://openimaginglab.github.io/AdaptiveISP/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。