端到端优化真实相机成像,修复镜头暗角与压缩冗余。
An End-to-End Real-World Camera Imaging Pipeline
- 全链路端到端设计,联合优化图像处理流程。
- 在真实数据集上实现最优码率-失真表现与更低延迟。
- 适合需要高画质与低延迟的移动相机系统开发。
近期神经相机成像管线取得显著进展,但真实场景中仍面临系统组件缺乏联合优化、计算冗余及镜头暗角等光学畸变问题。为此,我们提出端到端相机成像管线RealCamNet,以提升真实世界成像性能。该方法突破传统分阶段图像信号处理,采用全链路端到端架构,实现全流程联合优化,并恢复坐标相关畸变。RealCamNet专用于从RAW到RGB的高质量转换与紧凑图像压缩。我们深入分析了如渐晕、暗角等坐标依赖型光学畸变,设计了坐标感知畸变恢复(CADR)模块以修复此类问题;同时提出坐标无关映射压缩(CIMC)模块,实现色调映射与冗余信息压缩。现有数据集存在对齐偏差且条件过于理想化,难以支撑真实成像训练,因此我们构建了一个真实世界成像数据集。实验结果表明,RealCamNet在码率-失真性能上达到最优,且推理延迟更低。
原文摘要 · Abstract (English)
Recent advances in neural camera imaging pipelines have demonstrated notable progress. Nevertheless, the real-world imaging pipeline still faces challenges including the lack of joint optimization in system components, computational redundancies, and optical distortions such as lens shading.In light of this, we propose an end-to-end camera imaging pipeline (RealCamNet) to enhance real-world camera imaging performance. Our methodology diverges from conventional, fragmented multi-stage image signal processing towards end-to-end architecture. This architecture facilitates joint optimization across the full pipeline and the restoration of coordinate-biased distortions. RealCamNet is designed for high-quality conversion from RAW to RGB and compact image compression. Specifically, we deeply analyze coordinate-dependent optical distortions, e.g., vignetting and dark shading, and design a novel Coordinate-Aware Distortion Restoration (CADR) module to restore coordinate-biased distortions. Furthermore, we propose a Coordinate-Independent Mapping Compression (CIMC) module to implement tone mapping and redundant information compression. Existing datasets suffer from misalignment and overly idealized conditions, making them inadequate for training real-world imaging pipelines. Therefore, we collected a real-world imaging dataset. Experiment results show that RealCamNet achieves the best rate-distortion performance with lower inference latency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。