统一处理人眼与机器视觉的图像处理框架,兼顾美观与识别性能。
UniISP: A Unified ISP Framework for Both Human and Machine Vision

- 采用混合注意力模块和监督学习,生成既清晰又美观的图像。
- 引入特征适配器,提升下游任务的特征传递效率。
- 在多个数据集上表现优异,适合同时服务人眼与算法需求。
相较于RGB图像,原始传感器数据包含更丰富的信息,对复杂环境(如低光照)下的准确识别至关重要。传统图像信号处理(ISP)流程虽能生成符合人眼审美的RGB图像,但部分操作会引入压缩与信息损失。而在直接使用原始数据的计算机视觉任务中,现有方法多仅做少量ISP处理,导致输出难以可视化或不符合人类审美。本文提出UniISP,一种可同时满足人眼感知与机器视觉需求的统一ISP框架。通过设计混合注意力模块(HAM)并结合监督学习,确保生成图像具有视觉吸引力;同时引入特征适配器模块,有效将ISP阶段的信息特征传递至下游网络。大量实验表明,该方法在多种场景与数据集上均达到当前最优性能,验证了其泛化性与有效性。
原文摘要 · Abstract (English)
Compared to RGB images, raw sensor data provides a richer representation of information, which is crucial for accurate recognition, particularly under challenging conditions such as low-light environments. The traditional Image Signal Processing (ISP) pipeline generates visually pleasing RGB images for human perception through a series of steps, but some of these operations may adversely impact the information integrity by introducing compression and loss. Furthermore, in computer vision tasks that directly utilize raw camera data, most existing methods integrate minimal ISP processing with downstream networks, yet the resulting images are often difficult to visualize or do not align with human aesthetic preferences. This paper proposes UniISP, a novel ISP framework designed to simultaneously meet the requirements of both human visual perception and computer vision applications. By incorporating a carefully designed Hybrid Attention Module (HAM) and employing supervised learning, the proposed method ensures that the generated images are visually appealing. Additionally, a Feature Adapter module is introduced to effectively propagate informative features from the ISP stage to subsequent downstream networks. Extensive experiments demonstrate that our approach achieves state-of-the-art performance across various scenarios and multiple datasets, proving its generalizability and effectiveness.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。