小像素相机成像模糊，神经ISP能逆向修复光学缺陷，提升画质。
The Need for Neural ISP in the Small-Pixel Era: How Shrinking Pixels Push Optics to the Limit and Neural Restoration Pushes Back
- 用学习式神经ISP建模光学退化，反推传统流水线无法解决的模糊问题
- 0.35微米像素下分辨率达745 cycles/mm，比传统ISP提升2.5至3倍
- 适合手机高倍变焦镜头设计，尤其在低光多帧场景中表现突出
智能手机长焦摄像头正逼近'长焦物理墙'：当像素尺寸缩小至0.5微米以下时，光学系统受几何像差限制，导致分辨率提升收益递减。传统图像信号处理器（ISP）因采用局部、分阶段处理且缺乏点扩散函数（PSF）显式模型，无法消除此类像差。本文通过模拟典型长焦模组，在五种像素尺寸（0.35–0.75微米）下评估，保持每像素信噪比与衍射斑大小不变以隔离几何像差和空间采样影响。传统ISP随像素缩小改善有限，而神经ISP显著提升：在0.35微米时，垂直方向MTF50达745 cycles/mm，较传统ISP提高2.5–3倍；LPIPS从0.244降至0.151，而传统结果基本持平。在低信噪比场景（单帧15 dB，0.35微米），多帧神经ISP性能接近亮光单帧基准，传统多帧方案无明显改善——表明传统管线在小像素下瓶颈在于未校正的PSF模糊而非噪声。结果表明，神经ISP可通过对残余光学像差的纠正，使小像素设计成为高分辨率长焦模块的新范式。
原文摘要 · Abstract (English)
Smartphone telephoto cameras are approaching a "telephoto physics wall": as pixel pitches shrink toward sub-0.5 micron, the optics remain limited by geometric aberrations, leading to diminishing returns on resolution. Traditional Image Signal Processors (ISPs) cannot eliminate these aberrations, because they operate through local, stage-wise processing with no explicit model of the underlying point spread function (PSF). We demonstrate how a learning-based Neural ISP for image restoration, trained on the underlying degradations, inverts what stage-wise pipelines cannot, turning small-pixel designs into a net advantage. We investigate this through a controlled simulation of a representative telephoto module, evaluating five configurations (0.35--0.75 micron pixel pitch). The aperture is scaled proportionally to keep per-pixel SNR and diffraction spot size fixed, thereby isolating geometric aberration and spatial sampling. While the traditional ISP improves only modestly with smaller pixels, the Neural ISP scales substantially: at 0.35 micron} it reaches 745 cycles/mm MTF50 (vertical), a 2.5--3x resolution improvement over the traditional ISP, and LPIPS improves significantly from 0.244 to 0.151 while traditional results stay comparatively flat. In a low-SNR extension (15 dB per-frame bursts at 0.35 micron), a multi-frame Neural ISP recovers performance close to the bright-light single-frame baseline, whereas a multi-frame traditional ISP shows no meaningful improvement -- indicating that traditional pipelines at small pixels are bottlenecked by uncorrected PSF blur rather than by noise. These results point to a design philosophy in which Neural ISPs enable high-resolution telephoto modules by correcting residual optical aberrations rather than requiring increasingly complex optics.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。