用单次推理优化图像处理流水线，提升任务性能并降低成本
POS-ISP: Pipeline Optimization at the Sequence Level for Task-aware ISP
- 将模块序列与参数优化统一为全局序列预测问题
- 在多个下游任务上提升性能同时降低计算开销
- 适合需要高效定制化图像处理的视觉系统开发者
近期工作通过组合预定义模块并适配任务目标来优化图像信号处理（ISP）流水线。然而，联合优化模块序列与参数仍具挑战性。现有方法依赖神经架构搜索（NAS）或分步强化学习（RL），但NAS存在训练-推理不一致问题，分步RL则因逐阶段决策导致训练不稳定且计算开销高。本文提出POS-ISP，一种序列级强化学习框架，将模块化ISP优化建模为全局序列预测问题。该方法在单次前向传播中预测完整模块序列及其参数，并基于终端任务奖励进行优化，无需中间监督和冗余执行。在多个下游任务上的实验表明，POS-ISP在提升任务性能的同时降低了计算成本，凸显序列级优化在任务感知ISP中的稳定性和高效性。项目页面见：https://w1jyun.github.io/POS-ISP
原文摘要 · Abstract (English)
Recent work has explored optimizing image signal processing (ISP) pipelines for various tasks by composing predefined modules and adapting them to task-specific objectives. However, jointly optimizing module sequences and parameters remains challenging. Existing approaches rely on neural architecture search (NAS) or step-wise reinforcement learning (RL), but NAS suffers from a training-inference mismatch, while step-wise RL leads to unstable training and high computational overhead due to stage-wise decision-making. We propose POS-ISP, a sequence-level RL framework that formulates modular ISP optimization as a global sequence prediction problem. Our method predicts the entire module sequence and its parameters in a single forward pass and optimizes the pipeline using a terminal task reward, eliminating the need for intermediate supervision and redundant executions. Experiments across multiple downstream tasks show that POS-ISP improves task performance while reducing computational cost, highlighting sequence-level optimization as a stable and efficient paradigm for task-aware ISP. The project page is available at https://w1jyun.github.io/POS-ISP
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。